INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48884 - "GET /node/69e6cb6b60f5826562c15bc3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59376 - "GET /node/69e342d33745d1193f227a4b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59912 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:24:51,927 - lava_callback - INFO - Received callback for node 69f2a13ee2fa3d7693ba43f1 2026-05-02 00:24:51,933 - lava_callback - INFO - Starting async job submit for node 69f2a13ee2fa3d7693ba43f1 INFO: 172.20.0.1:41566 - "POST /node/69f2a13ee2fa3d7693ba43f1 HTTP/1.0" 202 Accepted 2026-05-02 00:24:51,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:24:51,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a13ee2fa3d7693ba43f1 HTTP/1.1" 200 None 2026-05-02 00:24:51,976 - lava_callback - DEBUG - Processing node 69f2a13ee2fa3d7693ba43f1: baseline-arm-clabbe 2026-05-02 00:24:51,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a13ee2fa3d7693ba43f1 2026-05-02 00:24:51,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:24:52,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a13ee2fa3d7693ba43f1 HTTP/1.1" 200 None 2026-05-02 00:24:52,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:24:52,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:24:52,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:24:52,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a13ee2fa3d7693ba43f1 HTTP/1.1" 200 None 2026-05-02 00:24:52,075 - lava_callback - ERROR - Error processing callback for node 69f2a13ee2fa3d7693ba43f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:25:31,842 - lava_callback - INFO - Received callback for node 69f2a179e2fa3d7693ba43ff INFO: 172.20.0.1:57620 - "POST /node/69f2a179e2fa3d7693ba43ff HTTP/1.0" 202 Accepted 2026-05-02 00:25:31,852 - lava_callback - INFO - Starting async job submit for node 69f2a179e2fa3d7693ba43ff 2026-05-02 00:25:31,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:25:31,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a179e2fa3d7693ba43ff HTTP/1.1" 200 None 2026-05-02 00:25:31,884 - lava_callback - DEBUG - Processing node 69f2a179e2fa3d7693ba43ff: baseline-arm-clabbe 2026-05-02 00:25:31,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a179e2fa3d7693ba43ff 2026-05-02 00:25:31,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:25:32,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a179e2fa3d7693ba43ff HTTP/1.1" 200 None 2026-05-02 00:25:32,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:25:32,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:25:32,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:25:32,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a179e2fa3d7693ba43ff HTTP/1.1" 200 None 2026-05-02 00:25:32,105 - lava_callback - ERROR - Error processing callback for node 69f2a179e2fa3d7693ba43ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33806 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:26:31,931 - lava_callback - INFO - Received callback for node 69f2a1b5e2fa3d7693ba440f INFO: 172.20.0.1:41254 - "POST /node/69f2a1b5e2fa3d7693ba440f HTTP/1.0" 202 Accepted 2026-05-02 00:26:31,940 - lava_callback - INFO - Starting async job submit for node 69f2a1b5e2fa3d7693ba440f 2026-05-02 00:26:31,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:26:31,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a1b5e2fa3d7693ba440f HTTP/1.1" 200 None 2026-05-02 00:26:31,966 - lava_callback - DEBUG - Processing node 69f2a1b5e2fa3d7693ba440f: baseline-arm-clabbe 2026-05-02 00:26:31,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a1b5e2fa3d7693ba440f 2026-05-02 00:26:31,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:26:32,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a1b5e2fa3d7693ba440f HTTP/1.1" 200 None 2026-05-02 00:26:32,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:26:32,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:26:32,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:26:32,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a1b5e2fa3d7693ba440f HTTP/1.1" 200 None 2026-05-02 00:26:32,047 - lava_callback - ERROR - Error processing callback for node 69f2a1b5e2fa3d7693ba440f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52430 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:27:11,925 - lava_callback - INFO - Received callback for node 69f2a1d9e2fa3d7693ba441c INFO: 172.20.0.1:52732 - "POST /node/69f2a1d9e2fa3d7693ba441c HTTP/1.0" 202 Accepted 2026-05-02 00:27:11,930 - lava_callback - INFO - Starting async job submit for node 69f2a1d9e2fa3d7693ba441c 2026-05-02 00:27:11,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:27:11,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a1d9e2fa3d7693ba441c HTTP/1.1" 200 None 2026-05-02 00:27:11,946 - lava_callback - DEBUG - Processing node 69f2a1d9e2fa3d7693ba441c: baseline-arm-clabbe 2026-05-02 00:27:11,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a1d9e2fa3d7693ba441c 2026-05-02 00:27:11,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:27:11,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a1d9e2fa3d7693ba441c HTTP/1.1" 200 None 2026-05-02 00:27:11,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:27:11,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:27:11,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:27:12,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a1d9e2fa3d7693ba441c HTTP/1.1" 200 None 2026-05-02 00:27:12,006 - lava_callback - ERROR - Error processing callback for node 69f2a1d9e2fa3d7693ba441c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52610 - "GET /node/69f24121f5ce4820821f063c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52614 - "GET /node/69e505e39a34686dd62eb1b8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37772 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:28:52,054 - lava_callback - INFO - Received callback for node 69f2a22de2fa3d7693ba4436 INFO: 172.20.0.1:44728 - "POST /node/69f2a22de2fa3d7693ba4436 HTTP/1.0" 202 Accepted 2026-05-02 00:28:52,061 - lava_callback - INFO - Starting async job submit for node 69f2a22de2fa3d7693ba4436 2026-05-02 00:28:52,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:28:52,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a22de2fa3d7693ba4436 HTTP/1.1" 200 None 2026-05-02 00:28:52,079 - lava_callback - DEBUG - Processing node 69f2a22de2fa3d7693ba4436: baseline-arm-clabbe 2026-05-02 00:28:52,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a22de2fa3d7693ba4436 2026-05-02 00:28:52,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:28:52,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a22de2fa3d7693ba4436 HTTP/1.1" 200 None 2026-05-02 00:28:52,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:28:52,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:28:52,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:28:52,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a22de2fa3d7693ba4436 HTTP/1.1" 200 None 2026-05-02 00:28:52,146 - lava_callback - ERROR - Error processing callback for node 69f2a22de2fa3d7693ba4436: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44062 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:29:52,123 - lava_callback - INFO - Received callback for node 69f2a26ae2fa3d7693ba444d INFO: 172.20.0.1:44736 - "POST /node/69f2a26ae2fa3d7693ba444d HTTP/1.0" 202 Accepted 2026-05-02 00:29:52,132 - lava_callback - INFO - Starting async job submit for node 69f2a26ae2fa3d7693ba444d 2026-05-02 00:29:52,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:29:52,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a26ae2fa3d7693ba444d HTTP/1.1" 200 None 2026-05-02 00:29:52,149 - lava_callback - DEBUG - Processing node 69f2a26ae2fa3d7693ba444d: baseline-arm-clabbe 2026-05-02 00:29:52,149 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a26ae2fa3d7693ba444d 2026-05-02 00:29:52,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:29:52,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a26ae2fa3d7693ba444d HTTP/1.1" 200 None 2026-05-02 00:29:52,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:29:52,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:29:52,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:29:52,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a26ae2fa3d7693ba444d HTTP/1.1" 200 None 2026-05-02 00:29:52,198 - lava_callback - ERROR - Error processing callback for node 69f2a26ae2fa3d7693ba444d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39558 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:35:52,366 - lava_callback - INFO - Received callback for node 69f2a3d2e2fa3d7693ba44e1 INFO: 172.20.0.1:44218 - "POST /node/69f2a3d2e2fa3d7693ba44e1 HTTP/1.0" 202 Accepted 2026-05-02 00:35:52,374 - lava_callback - INFO - Starting async job submit for node 69f2a3d2e2fa3d7693ba44e1 2026-05-02 00:35:52,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:35:52,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a3d2e2fa3d7693ba44e1 HTTP/1.1" 200 None 2026-05-02 00:35:52,391 - lava_callback - DEBUG - Processing node 69f2a3d2e2fa3d7693ba44e1: baseline-arm-clabbe 2026-05-02 00:35:52,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a3d2e2fa3d7693ba44e1 2026-05-02 00:35:52,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:35:52,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a3d2e2fa3d7693ba44e1 HTTP/1.1" 200 None 2026-05-02 00:35:52,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:35:52,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:35:52,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:35:52,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a3d2e2fa3d7693ba44e1 HTTP/1.1" 200 None 2026-05-02 00:35:52,457 - lava_callback - ERROR - Error processing callback for node 69f2a3d2e2fa3d7693ba44e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET /node/69e3bc877f5347282fbde9c6%5Cn HTTP/1.0" 405 Method Not Allowed 2026-05-02 00:36:50,248 - lava_callback - INFO - Received callback for node 69f2a413e2fa3d7693ba44f2 INFO: 172.20.0.1:38872 - "POST /node/69f2a413e2fa3d7693ba44f2 HTTP/1.0" 202 Accepted 2026-05-02 00:36:50,254 - lava_callback - INFO - Starting async job submit for node 69f2a413e2fa3d7693ba44f2 2026-05-02 00:36:50,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:36:50,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a413e2fa3d7693ba44f2 HTTP/1.1" 200 None 2026-05-02 00:36:50,269 - lava_callback - DEBUG - Processing node 69f2a413e2fa3d7693ba44f2: baseline-arm64-baylibre 2026-05-02 00:36:50,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a413e2fa3d7693ba44f2 2026-05-02 00:36:50,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:36:50,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a413e2fa3d7693ba44f2 HTTP/1.1" 200 None 2026-05-02 00:36:50,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:36:50,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:36:50,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:36:50,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a413e2fa3d7693ba44f2 HTTP/1.1" 200 None 2026-05-02 00:36:50,314 - lava_callback - ERROR - Error processing callback for node 69f2a413e2fa3d7693ba44f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42766 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:38:10,167 - lava_callback - INFO - Received callback for node 69f2a45ee2fa3d7693ba4519 INFO: 172.20.0.1:59856 - "POST /node/69f2a45ee2fa3d7693ba4519 HTTP/1.0" 202 Accepted 2026-05-02 00:38:10,172 - lava_callback - INFO - Starting async job submit for node 69f2a45ee2fa3d7693ba4519 2026-05-02 00:38:10,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:38:10,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a45ee2fa3d7693ba4519 HTTP/1.1" 200 None 2026-05-02 00:38:10,187 - lava_callback - DEBUG - Processing node 69f2a45ee2fa3d7693ba4519: baseline-arm64-baylibre 2026-05-02 00:38:10,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a45ee2fa3d7693ba4519 2026-05-02 00:38:10,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:38:10,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a45ee2fa3d7693ba4519 HTTP/1.1" 200 None 2026-05-02 00:38:10,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:38:10,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:38:10,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:38:10,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a45ee2fa3d7693ba4519 HTTP/1.1" 200 None 2026-05-02 00:38:10,232 - lava_callback - ERROR - Error processing callback for node 69f2a45ee2fa3d7693ba4519: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36968 - "GET /node/69f1539b1c6f6c1a9f7057dd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36976 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:39:10,210 - lava_callback - INFO - Received callback for node 69f2a4abe2fa3d7693ba453a INFO: 172.20.0.1:32944 - "POST /node/69f2a4abe2fa3d7693ba453a HTTP/1.0" 202 Accepted 2026-05-02 00:39:10,216 - lava_callback - INFO - Starting async job submit for node 69f2a4abe2fa3d7693ba453a 2026-05-02 00:39:10,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:39:10,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a4abe2fa3d7693ba453a HTTP/1.1" 200 None 2026-05-02 00:39:10,232 - lava_callback - DEBUG - Processing node 69f2a4abe2fa3d7693ba453a: baseline-arm64-baylibre 2026-05-02 00:39:10,232 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a4abe2fa3d7693ba453a 2026-05-02 00:39:10,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:39:10,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a4abe2fa3d7693ba453a HTTP/1.1" 200 None 2026-05-02 00:39:10,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:39:10,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:39:10,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:39:10,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a4abe2fa3d7693ba453a HTTP/1.1" 200 None 2026-05-02 00:39:10,283 - lava_callback - ERROR - Error processing callback for node 69f2a4abe2fa3d7693ba453a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET /node/697aacbf79032e239ba0f5dd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51106 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:40:50,393 - lava_callback - INFO - Received callback for node 69f2a505e2fa3d7693ba455c INFO: 172.20.0.1:53824 - "POST /node/69f2a505e2fa3d7693ba455c HTTP/1.0" 202 Accepted 2026-05-02 00:40:50,401 - lava_callback - INFO - Starting async job submit for node 69f2a505e2fa3d7693ba455c 2026-05-02 00:40:50,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:40:50,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a505e2fa3d7693ba455c HTTP/1.1" 200 None 2026-05-02 00:40:50,418 - lava_callback - DEBUG - Processing node 69f2a505e2fa3d7693ba455c: baseline-arm64-baylibre 2026-05-02 00:40:50,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a505e2fa3d7693ba455c 2026-05-02 00:40:50,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:40:50,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a505e2fa3d7693ba455c HTTP/1.1" 200 None 2026-05-02 00:40:50,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:40:50,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:40:50,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:40:50,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a505e2fa3d7693ba455c HTTP/1.1" 200 None 2026-05-02 00:40:50,482 - lava_callback - ERROR - Error processing callback for node 69f2a505e2fa3d7693ba455c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59136 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:42:10,450 - lava_callback - INFO - Received callback for node 69f2a552e2fa3d7693ba4586 INFO: 172.20.0.1:54418 - "POST /node/69f2a552e2fa3d7693ba4586 HTTP/1.0" 202 Accepted 2026-05-02 00:42:10,456 - lava_callback - INFO - Starting async job submit for node 69f2a552e2fa3d7693ba4586 2026-05-02 00:42:10,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:42:10,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a552e2fa3d7693ba4586 HTTP/1.1" 200 None 2026-05-02 00:42:10,471 - lava_callback - DEBUG - Processing node 69f2a552e2fa3d7693ba4586: baseline-arm64-baylibre 2026-05-02 00:42:10,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a552e2fa3d7693ba4586 2026-05-02 00:42:10,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:42:10,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a552e2fa3d7693ba4586 HTTP/1.1" 200 None 2026-05-02 00:42:10,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:42:10,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:42:10,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:42:10,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a552e2fa3d7693ba4586 HTTP/1.1" 200 None 2026-05-02 00:42:10,526 - lava_callback - ERROR - Error processing callback for node 69f2a552e2fa3d7693ba4586: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:43:10,497 - lava_callback - INFO - Received callback for node 69f2a59ce2fa3d7693ba45be INFO: 172.20.0.1:32948 - "POST /node/69f2a59ce2fa3d7693ba45be HTTP/1.0" 202 Accepted 2026-05-02 00:43:10,502 - lava_callback - INFO - Starting async job submit for node 69f2a59ce2fa3d7693ba45be 2026-05-02 00:43:10,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:10,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a59ce2fa3d7693ba45be HTTP/1.1" 200 None 2026-05-02 00:43:10,518 - lava_callback - DEBUG - Processing node 69f2a59ce2fa3d7693ba45be: baseline-arm64-baylibre 2026-05-02 00:43:10,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a59ce2fa3d7693ba45be 2026-05-02 00:43:10,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:10,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a59ce2fa3d7693ba45be HTTP/1.1" 200 None 2026-05-02 00:43:10,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:10,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:43:10,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:10,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a59ce2fa3d7693ba45be HTTP/1.1" 200 None 2026-05-02 00:43:10,593 - lava_callback - ERROR - Error processing callback for node 69f2a59ce2fa3d7693ba45be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43062 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:43:44,704 - lava_callback - INFO - Received callback for node 69f54844ed4ed40b02240349 INFO: 172.20.0.1:52376 - "POST /node/69f54844ed4ed40b02240349 HTTP/1.0" 202 Accepted 2026-05-02 00:43:44,710 - lava_callback - INFO - Starting async job submit for node 69f54844ed4ed40b02240349 2026-05-02 00:43:44,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:44,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54844ed4ed40b02240349 HTTP/1.1" 200 None 2026-05-02 00:43:44,758 - lava_callback - DEBUG - Processing node 69f54844ed4ed40b02240349: baseline-arm 2026-05-02 00:43:44,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:43:45,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:43:45,100 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f54844ed4ed40b02240349/log.txt.gz 2026-05-02 00:43:45,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:43:45,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:43:45,365 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f54844ed4ed40b02240349/lava-logs.yaml 2026-05-02 00:43:45,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:43:45,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:43:45,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f54844ed4ed40b02240349/lava_callback.json.gz 2026-05-02 00:43:45,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:45,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54844ed4ed40b02240349 HTTP/1.1" 200 None 2026-05-02 00:43:45,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:45,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54464ed4ed40b0223db05 HTTP/1.1" 200 None 2026-05-02 00:43:45,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:45,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54844ed4ed40b02240349 HTTP/1.1" 200 None 2026-05-02 00:43:45,545 - lava_callback - INFO - Completed processing callback for node 69f54844ed4ed40b02240349 INFO: 172.20.0.1:52386 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:43:52,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:43:52,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56076 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:47:53,010 - lava_callback - INFO - Received callback for node 69f2a6b4e2fa3d7693ba6d66 INFO: 172.20.0.1:41954 - "POST /node/69f2a6b4e2fa3d7693ba6d66 HTTP/1.0" 202 Accepted 2026-05-02 00:47:53,016 - lava_callback - INFO - Starting async job submit for node 69f2a6b4e2fa3d7693ba6d66 2026-05-02 00:47:53,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:47:53,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a6b4e2fa3d7693ba6d66 HTTP/1.1" 200 None 2026-05-02 00:47:53,035 - lava_callback - DEBUG - Processing node 69f2a6b4e2fa3d7693ba6d66: baseline-arm64-clabbe 2026-05-02 00:47:53,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a6b4e2fa3d7693ba6d66 2026-05-02 00:47:53,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:47:53,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a6b4e2fa3d7693ba6d66 HTTP/1.1" 200 None 2026-05-02 00:47:53,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:47:53,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:47:53,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:47:53,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a6b4e2fa3d7693ba6d66 HTTP/1.1" 200 None 2026-05-02 00:47:53,114 - lava_callback - ERROR - Error processing callback for node 69f2a6b4e2fa3d7693ba6d66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37030 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:48:37,612 - lava_callback - INFO - Received callback for node 69f549dded4ed40b0224042b INFO: 172.20.0.1:37034 - "POST /node/69f549dded4ed40b0224042b HTTP/1.0" 202 Accepted 2026-05-02 00:48:37,618 - lava_callback - INFO - Starting async job submit for node 69f549dded4ed40b0224042b 2026-05-02 00:48:37,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:48:37,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dded4ed40b0224042b HTTP/1.1" 200 None 2026-05-02 00:48:37,636 - lava_callback - DEBUG - Processing node 69f549dded4ed40b0224042b: baseline-arm 2026-05-02 00:48:37,642 - lava_callback - WARNING - Failed to upload log 2026-05-02 00:48:37,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:48:37,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:48:37,751 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dded4ed40b0224042b/lava-logs.yaml 2026-05-02 00:48:37,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:48:37,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:48:37,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dded4ed40b0224042b/lava_callback.json.gz 2026-05-02 00:48:37,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:48:37,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dded4ed40b0224042b HTTP/1.1" 200 None 2026-05-02 00:48:37,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:48:37,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:48:37,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:48:37,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549dded4ed40b0224042b HTTP/1.1" 200 None 2026-05-02 00:48:37,971 - lava_callback - INFO - Completed processing callback for node 69f549dded4ed40b0224042b INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:48:52,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:48:52,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52742 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:49:05,546 - lava_callback - INFO - Received callback for node 69f549dded4ed40b0224042e INFO: 172.20.0.1:37366 - "POST /node/69f549dded4ed40b0224042e HTTP/1.0" 202 Accepted 2026-05-02 00:49:05,594 - lava_callback - INFO - Starting async job submit for node 69f549dded4ed40b0224042e 2026-05-02 00:49:05,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:05,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dded4ed40b0224042e HTTP/1.1" 200 None 2026-05-02 00:49:05,637 - lava_callback - DEBUG - Processing node 69f549dded4ed40b0224042e: baseline-arm 2026-05-02 00:49:05,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:05,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:05,864 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dded4ed40b0224042e/log.txt.gz 2026-05-02 00:49:05,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:06,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:06,048 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dded4ed40b0224042e/lava-logs.yaml 2026-05-02 00:49:06,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:06,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:06,156 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dded4ed40b0224042e/lava_callback.json.gz 2026-05-02 00:49:06,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:06,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dded4ed40b0224042e HTTP/1.1" 200 None 2026-05-02 00:49:06,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:06,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:49:06,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:06,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549dded4ed40b0224042e HTTP/1.1" 200 None 2026-05-02 00:49:06,240 - lava_callback - INFO - Completed processing callback for node 69f549dded4ed40b0224042e INFO: 172.20.0.1:37382 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:49:09,629 - lava_callback - INFO - Received callback for node 69f549e0ed4ed40b02240437 INFO: 172.20.0.1:37394 - "POST /node/69f549e0ed4ed40b02240437 HTTP/1.0" 202 Accepted 2026-05-02 00:49:09,678 - lava_callback - INFO - Starting async job submit for node 69f549e0ed4ed40b02240437 2026-05-02 00:49:09,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:09,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549e0ed4ed40b02240437 HTTP/1.1" 200 None 2026-05-02 00:49:09,710 - lava_callback - DEBUG - Processing node 69f549e0ed4ed40b02240437: baseline-arm 2026-05-02 00:49:09,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:09,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:09,922 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f549e0ed4ed40b02240437/log.txt.gz 2026-05-02 00:49:09,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:10,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:10,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f549e0ed4ed40b02240437/lava-logs.yaml 2026-05-02 00:49:10,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:10,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:10,195 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f549e0ed4ed40b02240437/lava_callback.json.gz 2026-05-02 00:49:10,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:10,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549e0ed4ed40b02240437 HTTP/1.1" 200 None 2026-05-02 00:49:10,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:10,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:49:10,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:10,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549e0ed4ed40b02240437 HTTP/1.1" 200 None 2026-05-02 00:49:10,274 - lava_callback - INFO - Completed processing callback for node 69f549e0ed4ed40b02240437 2026-05-02 00:49:13,071 - lava_callback - INFO - Received callback for node 69f2a704e2fa3d7693ba6d84 INFO: 172.20.0.1:33110 - "POST /node/69f2a704e2fa3d7693ba6d84 HTTP/1.0" 202 Accepted 2026-05-02 00:49:13,077 - lava_callback - INFO - Starting async job submit for node 69f2a704e2fa3d7693ba6d84 2026-05-02 00:49:13,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:13,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a704e2fa3d7693ba6d84 HTTP/1.1" 200 None 2026-05-02 00:49:13,092 - lava_callback - DEBUG - Processing node 69f2a704e2fa3d7693ba6d84: baseline-arm64-clabbe 2026-05-02 00:49:13,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a704e2fa3d7693ba6d84 2026-05-02 00:49:13,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:13,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a704e2fa3d7693ba6d84 HTTP/1.1" 200 None 2026-05-02 00:49:13,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:13,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:49:13,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:13,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a704e2fa3d7693ba6d84 HTTP/1.1" 200 None 2026-05-02 00:49:13,144 - lava_callback - ERROR - Error processing callback for node 69f2a704e2fa3d7693ba6d84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52536 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:49:22,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:22,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38978 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:49:53,915 - lava_callback - INFO - Received callback for node 69f549dfed4ed40b02240434 INFO: 172.20.0.1:57600 - "POST /node/69f549dfed4ed40b02240434 HTTP/1.0" 202 Accepted 2026-05-02 00:49:53,921 - lava_callback - INFO - Starting async job submit for node 69f549dfed4ed40b02240434 2026-05-02 00:49:53,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:53,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dfed4ed40b02240434 HTTP/1.1" 200 None 2026-05-02 00:49:53,982 - lava_callback - DEBUG - Processing node 69f549dfed4ed40b02240434: baseline-arm 2026-05-02 00:49:54,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:54,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:54,276 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dfed4ed40b02240434/log.txt.gz 2026-05-02 00:49:54,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:54,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:54,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dfed4ed40b02240434/lava-logs.yaml 2026-05-02 00:49:54,628 - lava_callback - INFO - Received callback for node 69f549feed4ed40b022404bc 2026-05-02 00:49:54,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:54,681 - lava_callback - INFO - Starting async job submit for node 69f549feed4ed40b022404bc INFO: 172.20.0.1:57612 - "POST /node/69f549feed4ed40b022404bc HTTP/1.0" 202 Accepted 2026-05-02 00:49:54,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:54,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549feed4ed40b022404bc HTTP/1.1" 200 None 2026-05-02 00:49:54,728 - lava_callback - DEBUG - Processing node 69f549feed4ed40b022404bc: baseline-arm-pengutronix 2026-05-02 00:49:54,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:54,801 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dfed4ed40b02240434/lava_callback.json.gz 2026-05-02 00:49:54,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:54,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:54,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dfed4ed40b02240434 HTTP/1.1" 200 None 2026-05-02 00:49:54,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:54,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:49:54,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:54,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549dfed4ed40b02240434 HTTP/1.1" 200 None 2026-05-02 00:49:54,950 - lava_callback - INFO - Completed processing callback for node 69f549dfed4ed40b02240434 2026-05-02 00:49:54,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:54,997 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549feed4ed40b022404bc/log.txt.gz 2026-05-02 00:49:55,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:55,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:55,215 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549feed4ed40b022404bc/lava-logs.yaml 2026-05-02 00:49:55,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:49:55,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:49:55,296 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549feed4ed40b022404bc/lava_callback.json.gz 2026-05-02 00:49:55,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:55,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549feed4ed40b022404bc HTTP/1.1" 200 None 2026-05-02 00:49:55,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:55,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:49:55,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:55,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549feed4ed40b022404bc HTTP/1.1" 200 None 2026-05-02 00:49:55,373 - lava_callback - INFO - Completed processing callback for node 69f549feed4ed40b022404bc INFO: 172.20.0.1:57614 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:49:59,928 - lava_callback - INFO - Received callback for node 69f549fded4ed40b022404b9 INFO: 172.20.0.1:49886 - "POST /node/69f549fded4ed40b022404b9 HTTP/1.0" 202 Accepted 2026-05-02 00:49:59,934 - lava_callback - INFO - Starting async job submit for node 69f549fded4ed40b022404b9 2026-05-02 00:49:59,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:49:59,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549fded4ed40b022404b9 HTTP/1.1" 200 None 2026-05-02 00:49:59,966 - lava_callback - DEBUG - Processing node 69f549fded4ed40b022404b9: baseline-arm-pengutronix 2026-05-02 00:50:00,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:00,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:00,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549fded4ed40b022404b9/log.txt.gz 2026-05-02 00:50:00,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:00,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:00,466 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549fded4ed40b022404b9/lava-logs.yaml 2026-05-02 00:50:00,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:00,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:00,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549fded4ed40b022404b9/lava_callback.json.gz 2026-05-02 00:50:00,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:00,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549fded4ed40b022404b9 HTTP/1.1" 200 None 2026-05-02 00:50:00,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:00,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:00,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:00,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549fded4ed40b022404b9 HTTP/1.1" 200 None 2026-05-02 00:50:00,637 - lava_callback - INFO - Completed processing callback for node 69f549fded4ed40b022404b9 2026-05-02 00:50:01,033 - lava_callback - INFO - Received callback for node 69f549fbed4ed40b022404b6 INFO: 172.20.0.1:49888 - "POST /node/69f549fbed4ed40b022404b6 HTTP/1.0" 202 Accepted 2026-05-02 00:50:01,040 - lava_callback - INFO - Starting async job submit for node 69f549fbed4ed40b022404b6 2026-05-02 00:50:01,063 - lava_callback - INFO - Received callback for node 69f549dced4ed40b02240428 2026-05-02 00:50:01,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:49900 - "POST /node/69f549dced4ed40b02240428 HTTP/1.0" 202 Accepted 2026-05-02 00:50:01,070 - lava_callback - INFO - Starting async job submit for node 69f549dced4ed40b02240428 2026-05-02 00:50:01,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:01,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549fbed4ed40b022404b6 HTTP/1.1" 200 None 2026-05-02 00:50:01,105 - lava_callback - DEBUG - Processing node 69f549fbed4ed40b022404b6: baseline-arm-pengutronix 2026-05-02 00:50:01,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dced4ed40b02240428 HTTP/1.1" 200 None 2026-05-02 00:50:01,174 - lava_callback - DEBUG - Processing node 69f549dced4ed40b02240428: baseline-arm 2026-05-02 00:50:01,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:01,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:01,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:01,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:01,619 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dced4ed40b02240428/log.txt.gz 2026-05-02 00:50:01,619 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549fbed4ed40b022404b6/log.txt.gz 2026-05-02 00:50:01,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:01,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:01,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:01,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549fbed4ed40b022404b6/lava-logs.yaml 2026-05-02 00:50:01,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:01,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:01,992 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dced4ed40b02240428/lava-logs.yaml 2026-05-02 00:50:01,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:02,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:02,061 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f549fbed4ed40b022404b6/lava_callback.json.gz 2026-05-02 00:50:02,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:02,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:02,102 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f549dced4ed40b02240428/lava_callback.json.gz 2026-05-02 00:50:02,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549fbed4ed40b022404b6 HTTP/1.1" 200 None 2026-05-02 00:50:02,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:02,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:02,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:02,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549dced4ed40b02240428 HTTP/1.1" 200 None 2026-05-02 00:50:02,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:02,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:02,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:02,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:02,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549fbed4ed40b022404b6 HTTP/1.1" 200 None 2026-05-02 00:50:02,164 - lava_callback - INFO - Completed processing callback for node 69f549fbed4ed40b022404b6 2026-05-02 00:50:02,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549dced4ed40b02240428 HTTP/1.1" 200 None 2026-05-02 00:50:02,204 - lava_callback - INFO - Completed processing callback for node 69f549dced4ed40b02240428 2026-05-02 00:50:05,873 - lava_callback - INFO - Received callback for node 69f549ebed4ed40b02240479 INFO: 172.20.0.1:49902 - "POST /node/69f549ebed4ed40b02240479 HTTP/1.0" 202 Accepted 2026-05-02 00:50:05,879 - lava_callback - INFO - Starting async job submit for node 69f549ebed4ed40b02240479 2026-05-02 00:50:05,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:05,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549ebed4ed40b02240479 HTTP/1.1" 200 None 2026-05-02 00:50:05,915 - lava_callback - DEBUG - Processing node 69f549ebed4ed40b02240479: baseline-arm-broonie 2026-05-02 00:50:06,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:06,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:06,222 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549ebed4ed40b02240479/log.txt.gz 2026-05-02 00:50:06,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:06,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:06,531 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549ebed4ed40b02240479/lava-logs.yaml 2026-05-02 00:50:06,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49912 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:50:06,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:06,641 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549ebed4ed40b02240479/lava_callback.json.gz 2026-05-02 00:50:06,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:06,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549ebed4ed40b02240479 HTTP/1.1" 200 None 2026-05-02 00:50:06,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:06,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:06,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:06,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549ebed4ed40b02240479 HTTP/1.1" 200 None 2026-05-02 00:50:06,725 - lava_callback - INFO - Completed processing callback for node 69f549ebed4ed40b02240479 2026-05-02 00:50:07,370 - lava_callback - INFO - Received callback for node 69f549eced4ed40b0224047c INFO: 172.20.0.1:49924 - "POST /node/69f549eced4ed40b0224047c HTTP/1.0" 202 Accepted 2026-05-02 00:50:07,376 - lava_callback - INFO - Starting async job submit for node 69f549eced4ed40b0224047c 2026-05-02 00:50:07,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:07,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549eced4ed40b0224047c HTTP/1.1" 200 None 2026-05-02 00:50:07,411 - lava_callback - DEBUG - Processing node 69f549eced4ed40b0224047c: baseline-arm-broonie 2026-05-02 00:50:07,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:07,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:07,738 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eced4ed40b0224047c/log.txt.gz 2026-05-02 00:50:07,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:08,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:08,055 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eced4ed40b0224047c/lava-logs.yaml 2026-05-02 00:50:08,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:08,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:08,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eced4ed40b0224047c/lava_callback.json.gz 2026-05-02 00:50:08,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:08,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549eced4ed40b0224047c HTTP/1.1" 200 None 2026-05-02 00:50:08,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:08,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:08,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:08,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549eced4ed40b0224047c HTTP/1.1" 200 None 2026-05-02 00:50:08,279 - lava_callback - INFO - Completed processing callback for node 69f549eced4ed40b0224047c INFO: 172.20.0.1:41636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54556 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:50:22,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:22,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:54570 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:50:30,847 - lava_callback - INFO - Received callback for node 69f549eeed4ed40b0224047f INFO: 172.20.0.1:36926 - "POST /node/69f549eeed4ed40b0224047f HTTP/1.0" 202 Accepted 2026-05-02 00:50:30,853 - lava_callback - INFO - Starting async job submit for node 69f549eeed4ed40b0224047f 2026-05-02 00:50:30,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:30,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549eeed4ed40b0224047f HTTP/1.1" 200 None 2026-05-02 00:50:30,896 - lava_callback - DEBUG - Processing node 69f549eeed4ed40b0224047f: baseline-arm-broonie 2026-05-02 00:50:31,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:31,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:31,223 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eeed4ed40b0224047f/log.txt.gz 2026-05-02 00:50:31,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:31,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:31,513 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eeed4ed40b0224047f/lava-logs.yaml 2026-05-02 00:50:31,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:31,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:31,618 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eeed4ed40b0224047f/lava_callback.json.gz 2026-05-02 00:50:31,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:31,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549eeed4ed40b0224047f HTTP/1.1" 200 None 2026-05-02 00:50:31,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:31,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:31,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:31,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549eeed4ed40b0224047f HTTP/1.1" 200 None 2026-05-02 00:50:31,722 - lava_callback - INFO - Completed processing callback for node 69f549eeed4ed40b0224047f 2026-05-02 00:50:33,141 - lava_callback - INFO - Received callback for node 69f2a752e2fa3d7693ba6daf INFO: 172.20.0.1:36932 - "POST /node/69f2a752e2fa3d7693ba6daf HTTP/1.0" 202 Accepted 2026-05-02 00:50:33,147 - lava_callback - INFO - Starting async job submit for node 69f2a752e2fa3d7693ba6daf 2026-05-02 00:50:33,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:33,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a752e2fa3d7693ba6daf HTTP/1.1" 200 None 2026-05-02 00:50:33,161 - lava_callback - DEBUG - Processing node 69f2a752e2fa3d7693ba6daf: baseline-arm64-clabbe 2026-05-02 00:50:33,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a752e2fa3d7693ba6daf 2026-05-02 00:50:33,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:33,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a752e2fa3d7693ba6daf HTTP/1.1" 200 None 2026-05-02 00:50:33,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:33,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:50:33,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:33,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a752e2fa3d7693ba6daf HTTP/1.1" 200 None 2026-05-02 00:50:33,208 - lava_callback - ERROR - Error processing callback for node 69f2a752e2fa3d7693ba6daf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:50:34,908 - lava_callback - INFO - Received callback for node 69f549f8ed4ed40b022404a8 INFO: 172.20.0.1:36934 - "POST /node/69f549f8ed4ed40b022404a8 HTTP/1.0" 202 Accepted 2026-05-02 00:50:34,914 - lava_callback - INFO - Starting async job submit for node 69f549f8ed4ed40b022404a8 2026-05-02 00:50:34,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:34,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549f8ed4ed40b022404a8 HTTP/1.1" 200 None 2026-05-02 00:50:34,951 - lava_callback - DEBUG - Processing node 69f549f8ed4ed40b022404a8: baseline-arm-kontron 2026-05-02 00:50:35,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:35,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:35,283 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f549f8ed4ed40b022404a8/log.txt.gz 2026-05-02 00:50:35,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:35,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:35,566 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f549f8ed4ed40b022404a8/lava-logs.yaml 2026-05-02 00:50:35,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:35,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:35,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f549f8ed4ed40b022404a8/lava_callback.json.gz 2026-05-02 00:50:35,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:35,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549f8ed4ed40b022404a8 HTTP/1.1" 200 None 2026-05-02 00:50:35,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:35,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:35,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:35,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549f8ed4ed40b022404a8 HTTP/1.1" 200 None 2026-05-02 00:50:35,749 - lava_callback - INFO - Completed processing callback for node 69f549f8ed4ed40b022404a8 INFO: 172.20.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52258 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:50:51,746 - lava_callback - INFO - Received callback for node 69f549eaed4ed40b02240474 INFO: 172.20.0.1:36732 - "POST /node/69f549eaed4ed40b02240474 HTTP/1.0" 202 Accepted 2026-05-02 00:50:51,753 - lava_callback - INFO - Starting async job submit for node 69f549eaed4ed40b02240474 2026-05-02 00:50:51,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:51,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549eaed4ed40b02240474 HTTP/1.1" 200 None 2026-05-02 00:50:51,793 - lava_callback - DEBUG - Processing node 69f549eaed4ed40b02240474: baseline-arm-broonie 2026-05-02 00:50:52,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:52,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:52,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 00:50:52,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:52,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eaed4ed40b02240474/log.txt.gz 2026-05-02 00:50:52,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:52,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:52,479 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eaed4ed40b02240474/lava-logs.yaml 2026-05-02 00:50:52,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:52,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:52,593 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f549eaed4ed40b02240474/lava_callback.json.gz 2026-05-02 00:50:52,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:52,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549eaed4ed40b02240474 HTTP/1.1" 200 None 2026-05-02 00:50:52,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:52,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:52,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:52,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549eaed4ed40b02240474 HTTP/1.1" 200 None 2026-05-02 00:50:52,679 - lava_callback - INFO - Completed processing callback for node 69f549eaed4ed40b02240474 2026-05-02 00:50:54,624 - lava_callback - INFO - Received callback for node 69f549f9ed4ed40b022404ab INFO: 172.20.0.1:36736 - "POST /node/69f549f9ed4ed40b022404ab HTTP/1.0" 202 Accepted 2026-05-02 00:50:54,630 - lava_callback - INFO - Starting async job submit for node 69f549f9ed4ed40b022404ab 2026-05-02 00:50:54,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:54,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549f9ed4ed40b022404ab HTTP/1.1" 200 None 2026-05-02 00:50:54,667 - lava_callback - DEBUG - Processing node 69f549f9ed4ed40b022404ab: baseline-arm-kontron 2026-05-02 00:50:54,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:54,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:54,937 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f549f9ed4ed40b022404ab/log.txt.gz 2026-05-02 00:50:55,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:55,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:55,212 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f549f9ed4ed40b022404ab/lava-logs.yaml 2026-05-02 00:50:55,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:50:55,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:50:55,324 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f549f9ed4ed40b022404ab/lava_callback.json.gz 2026-05-02 00:50:55,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:55,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549f9ed4ed40b022404ab HTTP/1.1" 200 None 2026-05-02 00:50:55,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:55,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:50:55,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:50:55,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549f9ed4ed40b022404ab HTTP/1.1" 200 None 2026-05-02 00:50:55,404 - lava_callback - INFO - Completed processing callback for node 69f549f9ed4ed40b022404ab INFO: 172.20.0.1:36740 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:51:01,484 - lava_callback - INFO - Received callback for node 69f54a65ed4ed40b022405a6 INFO: 172.20.0.1:34418 - "POST /node/69f54a65ed4ed40b022405a6 HTTP/1.0" 202 Accepted 2026-05-02 00:51:01,490 - lava_callback - INFO - Starting async job submit for node 69f54a65ed4ed40b022405a6 2026-05-02 00:51:01,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:01,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a65ed4ed40b022405a6 HTTP/1.1" 200 None 2026-05-02 00:51:01,511 - lava_callback - DEBUG - Processing node 69f54a65ed4ed40b022405a6: baseline-x86 2026-05-02 00:51:01,516 - lava_callback - WARNING - Failed to upload log 2026-05-02 00:51:01,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:01,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:01,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a65ed4ed40b022405a6/lava-logs.yaml 2026-05-02 00:51:01,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:01,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:01,692 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a65ed4ed40b022405a6/lava_callback.json.gz 2026-05-02 00:51:01,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:01,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a65ed4ed40b022405a6 HTTP/1.1" 200 None 2026-05-02 00:51:01,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:01,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54482ed4ed40b0223db29 HTTP/1.1" 200 None 2026-05-02 00:51:01,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:01,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a65ed4ed40b022405a6 HTTP/1.1" 200 None 2026-05-02 00:51:01,743 - lava_callback - INFO - Completed processing callback for node 69f54a65ed4ed40b022405a6 INFO: 172.20.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53584 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:51:21,874 - lava_callback - INFO - Received callback for node 69f54a67ed4ed40b022405af INFO: 172.20.0.1:53596 - "POST /node/69f54a67ed4ed40b022405af HTTP/1.0" 202 Accepted 2026-05-02 00:51:21,881 - lava_callback - INFO - Starting async job submit for node 69f54a67ed4ed40b022405af 2026-05-02 00:51:21,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:21,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a67ed4ed40b022405af HTTP/1.1" 200 None 2026-05-02 00:51:21,920 - lava_callback - DEBUG - Processing node 69f54a67ed4ed40b022405af: baseline-x86 2026-05-02 00:51:22,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:22,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:22,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a67ed4ed40b022405af/log.txt.gz 2026-05-02 00:51:22,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:22,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 00:51:22,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:22,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:22,434 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a67ed4ed40b022405af/lava-logs.yaml 2026-05-02 00:51:22,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:22,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:22,610 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a67ed4ed40b022405af/lava_callback.json.gz 2026-05-02 00:51:22,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:22,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a67ed4ed40b022405af HTTP/1.1" 200 None 2026-05-02 00:51:22,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:22,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54482ed4ed40b0223db29 HTTP/1.1" 200 None 2026-05-02 00:51:22,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:22,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a67ed4ed40b022405af HTTP/1.1" 200 None 2026-05-02 00:51:22,690 - lava_callback - INFO - Completed processing callback for node 69f54a67ed4ed40b022405af INFO: 172.20.0.1:53604 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:51:30,133 - lava_callback - INFO - Received callback for node 69f54a15ed4ed40b0224050e INFO: 172.20.0.1:50852 - "POST /node/69f54a15ed4ed40b0224050e HTTP/1.0" 202 Accepted 2026-05-02 00:51:30,140 - lava_callback - INFO - Starting async job submit for node 69f54a15ed4ed40b0224050e 2026-05-02 00:51:30,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:30,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a15ed4ed40b0224050e HTTP/1.1" 200 None 2026-05-02 00:51:30,180 - lava_callback - DEBUG - Processing node 69f54a15ed4ed40b0224050e: ltp-dio 2026-05-02 00:51:30,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:30,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:30,654 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69f54a15ed4ed40b0224050e/log.txt.gz 2026-05-02 00:51:30,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:31,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:31,078 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69f54a15ed4ed40b0224050e/lava-logs.yaml 2026-05-02 00:51:31,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:31,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:31,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69f54a15ed4ed40b0224050e/lava_callback.json.gz 2026-05-02 00:51:31,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:31,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a15ed4ed40b0224050e HTTP/1.1" 200 None 2026-05-02 00:51:31,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:31,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:51:31,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:31,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a15ed4ed40b0224050e HTTP/1.1" 200 None 2026-05-02 00:51:31,279 - lava_callback - INFO - Completed processing callback for node 69f54a15ed4ed40b0224050e INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:51:52,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:52,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 00:51:53,160 - lava_callback - INFO - Received callback for node 69f2a79ee2fa3d7693ba6dca INFO: 172.20.0.1:36218 - "POST /node/69f2a79ee2fa3d7693ba6dca HTTP/1.0" 202 Accepted 2026-05-02 00:51:53,166 - lava_callback - INFO - Starting async job submit for node 69f2a79ee2fa3d7693ba6dca 2026-05-02 00:51:53,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:53,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a79ee2fa3d7693ba6dca HTTP/1.1" 200 None 2026-05-02 00:51:53,185 - lava_callback - DEBUG - Processing node 69f2a79ee2fa3d7693ba6dca: baseline-arm64-clabbe 2026-05-02 00:51:53,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a79ee2fa3d7693ba6dca 2026-05-02 00:51:53,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:53,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a79ee2fa3d7693ba6dca HTTP/1.1" 200 None 2026-05-02 00:51:53,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:53,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:51:53,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:53,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a79ee2fa3d7693ba6dca HTTP/1.1" 200 None 2026-05-02 00:51:53,254 - lava_callback - ERROR - Error processing callback for node 69f2a79ee2fa3d7693ba6dca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36230 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:51:56,932 - lava_callback - INFO - Received callback for node 69f54a66ed4ed40b022405a9 INFO: 172.20.0.1:36246 - "POST /node/69f54a66ed4ed40b022405a9 HTTP/1.0" 202 Accepted 2026-05-02 00:51:56,941 - lava_callback - INFO - Starting async job submit for node 69f54a66ed4ed40b022405a9 2026-05-02 00:51:56,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:56,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a66ed4ed40b022405a9 HTTP/1.1" 200 None 2026-05-02 00:51:56,993 - lava_callback - DEBUG - Processing node 69f54a66ed4ed40b022405a9: baseline-x86 2026-05-02 00:51:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:57,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:57,489 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a66ed4ed40b022405a9/log.txt.gz 2026-05-02 00:51:57,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:57,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:57,904 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a66ed4ed40b022405a9/lava-logs.yaml 2026-05-02 00:51:57,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:51:57,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:51:58,001 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a66ed4ed40b022405a9/lava_callback.json.gz 2026-05-02 00:51:58,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:58,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a66ed4ed40b022405a9 HTTP/1.1" 200 None 2026-05-02 00:51:58,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:58,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54482ed4ed40b0223db29 HTTP/1.1" 200 None 2026-05-02 00:51:58,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:51:58,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a66ed4ed40b022405a9 HTTP/1.1" 200 None 2026-05-02 00:51:58,094 - lava_callback - INFO - Completed processing callback for node 69f54a66ed4ed40b022405a9 INFO: 172.20.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38530 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:52:18,955 - lava_callback - INFO - Received callback for node 69f54a67ed4ed40b022405ac INFO: 172.20.0.1:38536 - "POST /node/69f54a67ed4ed40b022405ac HTTP/1.0" 202 Accepted 2026-05-02 00:52:18,962 - lava_callback - INFO - Starting async job submit for node 69f54a67ed4ed40b022405ac 2026-05-02 00:52:18,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:19,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a67ed4ed40b022405ac HTTP/1.1" 200 None 2026-05-02 00:52:19,011 - lava_callback - DEBUG - Processing node 69f54a67ed4ed40b022405ac: baseline-x86 2026-05-02 00:52:19,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:52:19,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:52:19,444 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a67ed4ed40b022405ac/log.txt.gz 2026-05-02 00:52:19,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:52:19,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:52:19,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a67ed4ed40b022405ac/lava-logs.yaml 2026-05-02 00:52:19,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:52:19,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:52:19,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a67ed4ed40b022405ac/lava_callback.json.gz 2026-05-02 00:52:19,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:20,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a67ed4ed40b022405ac HTTP/1.1" 200 None 2026-05-02 00:52:20,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:20,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54482ed4ed40b0223db29 HTTP/1.1" 200 None 2026-05-02 00:52:20,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:20,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a67ed4ed40b022405ac HTTP/1.1" 200 None 2026-05-02 00:52:20,073 - lava_callback - INFO - Completed processing callback for node 69f54a67ed4ed40b022405ac INFO: 172.20.0.1:58880 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:52:22,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:22,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:52:28,035 - lava_callback - INFO - Received callback for node 69f54a69ed4ed40b022405b2 INFO: 172.20.0.1:58902 - "POST /node/69f54a69ed4ed40b022405b2 HTTP/1.0" 202 Accepted 2026-05-02 00:52:28,041 - lava_callback - INFO - Starting async job submit for node 69f54a69ed4ed40b022405b2 2026-05-02 00:52:28,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:28,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a69ed4ed40b022405b2 HTTP/1.1" 200 None 2026-05-02 00:52:28,078 - lava_callback - DEBUG - Processing node 69f54a69ed4ed40b022405b2: baseline-x86 2026-05-02 00:52:28,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:52:28,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:52:28,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a69ed4ed40b022405b2/log.txt.gz 2026-05-02 00:52:28,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:52:29,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:52:29,160 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a69ed4ed40b022405b2/lava-logs.yaml 2026-05-02 00:52:29,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:52:29,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:52:29,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f54a69ed4ed40b022405b2/lava_callback.json.gz 2026-05-02 00:52:29,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:29,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a69ed4ed40b022405b2 HTTP/1.1" 200 None 2026-05-02 00:52:29,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:29,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54482ed4ed40b0223db29 HTTP/1.1" 200 None 2026-05-02 00:52:29,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:29,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a69ed4ed40b022405b2 HTTP/1.1" 200 None 2026-05-02 00:52:29,380 - lava_callback - INFO - Completed processing callback for node 69f54a69ed4ed40b022405b2 INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53416 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:52:52,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:52:52,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:53:13,252 - lava_callback - INFO - Received callback for node 69f2a7eae2fa3d7693ba6def INFO: 172.20.0.1:52952 - "POST /node/69f2a7eae2fa3d7693ba6def HTTP/1.0" 202 Accepted 2026-05-02 00:53:13,258 - lava_callback - INFO - Starting async job submit for node 69f2a7eae2fa3d7693ba6def 2026-05-02 00:53:13,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:13,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a7eae2fa3d7693ba6def HTTP/1.1" 200 None 2026-05-02 00:53:13,279 - lava_callback - DEBUG - Processing node 69f2a7eae2fa3d7693ba6def: baseline-arm64-clabbe 2026-05-02 00:53:13,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a7eae2fa3d7693ba6def 2026-05-02 00:53:13,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:13,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a7eae2fa3d7693ba6def HTTP/1.1" 200 None 2026-05-02 00:53:13,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:13,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:53:13,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:13,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a7eae2fa3d7693ba6def HTTP/1.1" 200 None 2026-05-02 00:53:13,335 - lava_callback - ERROR - Error processing callback for node 69f2a7eae2fa3d7693ba6def: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:53:25,754 - lava_callback - INFO - Received callback for node 69f54a0eed4ed40b022404fa INFO: 172.20.0.1:37024 - "POST /node/69f54a0eed4ed40b022404fa HTTP/1.0" 202 Accepted 2026-05-02 00:53:25,761 - lava_callback - INFO - Starting async job submit for node 69f54a0eed4ed40b022404fa 2026-05-02 00:53:25,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:25,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a0eed4ed40b022404fa HTTP/1.1" 200 None 2026-05-02 00:53:25,800 - lava_callback - DEBUG - Processing node 69f54a0eed4ed40b022404fa: ltp-cap_bounds 2026-05-02 00:53:26,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:53:26,270 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:53:26,271 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69f54a0eed4ed40b022404fa/log.txt.gz 2026-05-02 00:53:26,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:37038 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:53:26,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:53:26,622 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69f54a0eed4ed40b022404fa/lava-logs.yaml 2026-05-02 00:53:26,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:53:26,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:53:26,748 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69f54a0eed4ed40b022404fa/lava_callback.json.gz 2026-05-02 00:53:26,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:26,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a0eed4ed40b022404fa HTTP/1.1" 200 None 2026-05-02 00:53:26,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:26,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:53:26,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:26,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a0eed4ed40b022404fa HTTP/1.1" 200 None 2026-05-02 00:53:26,828 - lava_callback - INFO - Completed processing callback for node 69f54a0eed4ed40b022404fa INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:53:51,079 - lava_callback - INFO - Received callback for node 69f54a20ed4ed40b02240526 INFO: 172.20.0.1:51130 - "POST /node/69f54a20ed4ed40b02240526 HTTP/1.0" 202 Accepted 2026-05-02 00:53:51,086 - lava_callback - INFO - Starting async job submit for node 69f54a20ed4ed40b02240526 2026-05-02 00:53:51,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:51,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a20ed4ed40b02240526 HTTP/1.1" 200 None 2026-05-02 00:53:51,126 - lava_callback - DEBUG - Processing node 69f54a20ed4ed40b02240526: ltp-syscalls 2026-05-02 00:53:51,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:53:51,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:53:51,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69f54a20ed4ed40b02240526/log.txt.gz 2026-05-02 00:53:52,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:52,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:53:52,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 00:53:52,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:53:52,632 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69f54a20ed4ed40b02240526/lava-logs.yaml 2026-05-02 00:53:52,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:53:52,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:53:52,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69f54a20ed4ed40b02240526/lava_callback.json.gz 2026-05-02 00:53:52,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:52,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a20ed4ed40b02240526 HTTP/1.1" 200 None 2026-05-02 00:53:52,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:52,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:53:52,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:53:52,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a20ed4ed40b02240526 HTTP/1.1" 200 None 2026-05-02 00:53:52,845 - lava_callback - INFO - Completed processing callback for node 69f54a20ed4ed40b02240526 INFO: 172.20.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34144 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:54:11,193 - lava_callback - INFO - Received callback for node 69f54a10ed4ed40b022404fd INFO: 172.20.0.1:35074 - "POST /node/69f54a10ed4ed40b022404fd HTTP/1.0" 202 Accepted 2026-05-02 00:54:11,200 - lava_callback - INFO - Starting async job submit for node 69f54a10ed4ed40b022404fd 2026-05-02 00:54:11,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:11,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a10ed4ed40b022404fd HTTP/1.1" 200 None 2026-05-02 00:54:11,238 - lava_callback - DEBUG - Processing node 69f54a10ed4ed40b022404fd: ltp-containers 2026-05-02 00:54:11,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:54:11,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:54:11,778 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69f54a10ed4ed40b022404fd/log.txt.gz 2026-05-02 00:54:12,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:54:12,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:54:12,167 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69f54a10ed4ed40b022404fd/lava-logs.yaml 2026-05-02 00:54:12,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:54:12,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:54:12,292 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69f54a10ed4ed40b022404fd/lava_callback.json.gz 2026-05-02 00:54:12,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:12,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a10ed4ed40b022404fd HTTP/1.1" 200 None 2026-05-02 00:54:12,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:12,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:54:12,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:12,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a10ed4ed40b022404fd HTTP/1.1" 200 None 2026-05-02 00:54:12,366 - lava_callback - INFO - Completed processing callback for node 69f54a10ed4ed40b022404fd INFO: 172.20.0.1:35084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35728 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:54:22,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:22,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54772 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:54:53,380 - lava_callback - INFO - Received callback for node 69f2a851e2fa3d7693ba6e13 INFO: 172.20.0.1:59464 - "POST /node/69f2a851e2fa3d7693ba6e13 HTTP/1.0" 202 Accepted 2026-05-02 00:54:53,385 - lava_callback - INFO - Starting async job submit for node 69f2a851e2fa3d7693ba6e13 2026-05-02 00:54:53,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:53,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a851e2fa3d7693ba6e13 HTTP/1.1" 200 None 2026-05-02 00:54:53,401 - lava_callback - DEBUG - Processing node 69f2a851e2fa3d7693ba6e13: baseline-arm64-clabbe 2026-05-02 00:54:53,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a851e2fa3d7693ba6e13 2026-05-02 00:54:53,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:53,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a851e2fa3d7693ba6e13 HTTP/1.1" 200 None 2026-05-02 00:54:53,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:53,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:54:53,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:53,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a851e2fa3d7693ba6e13 HTTP/1.1" 200 None 2026-05-02 00:54:53,457 - lava_callback - ERROR - Error processing callback for node 69f2a851e2fa3d7693ba6e13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:54:57,284 - lava_callback - INFO - Received callback for node 69f54b4ded4ed40b02240681 INFO: 172.20.0.1:59486 - "POST /node/69f54b4ded4ed40b02240681 HTTP/1.0" 202 Accepted 2026-05-02 00:54:57,294 - lava_callback - INFO - Starting async job submit for node 69f54b4ded4ed40b02240681 2026-05-02 00:54:57,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:57,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54b4ded4ed40b02240681 HTTP/1.1" 200 None 2026-05-02 00:54:57,313 - lava_callback - DEBUG - Processing node 69f54b4ded4ed40b02240681: baseline-arm 2026-05-02 00:54:57,321 - lava_callback - WARNING - Failed to upload log 2026-05-02 00:54:57,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:54:57,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:54:57,418 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f54b4ded4ed40b02240681/lava-logs.yaml 2026-05-02 00:54:57,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:54:57,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:54:57,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f54b4ded4ed40b02240681/lava_callback.json.gz 2026-05-02 00:54:57,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:57,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54b4ded4ed40b02240681 HTTP/1.1" 200 None 2026-05-02 00:54:57,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:57,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 00:54:57,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:54:57,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54b4ded4ed40b02240681 HTTP/1.1" 200 None 2026-05-02 00:54:57,573 - lava_callback - INFO - Completed processing callback for node 69f54b4ded4ed40b02240681 INFO: 172.20.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:55:22,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:55:22,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47836 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:55:53,386 - lava_callback - INFO - Received callback for node 69f2a890e2fa3d7693ba6e31 INFO: 172.20.0.1:60310 - "POST /node/69f2a890e2fa3d7693ba6e31 HTTP/1.0" 202 Accepted 2026-05-02 00:55:53,396 - lava_callback - INFO - Starting async job submit for node 69f2a890e2fa3d7693ba6e31 2026-05-02 00:55:53,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:55:53,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a890e2fa3d7693ba6e31 HTTP/1.1" 200 None 2026-05-02 00:55:53,425 - lava_callback - DEBUG - Processing node 69f2a890e2fa3d7693ba6e31: baseline-arm-clabbe 2026-05-02 00:55:53,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a890e2fa3d7693ba6e31 2026-05-02 00:55:53,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:55:53,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a890e2fa3d7693ba6e31 HTTP/1.1" 200 None 2026-05-02 00:55:53,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:55:53,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 00:55:53,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:55:53,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a890e2fa3d7693ba6e31 HTTP/1.1" 200 None 2026-05-02 00:55:53,515 - lava_callback - ERROR - Error processing callback for node 69f2a890e2fa3d7693ba6e31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:56:53,452 - lava_callback - INFO - Received callback for node 69f2a8c4e2fa3d7693ba6e85 INFO: 172.20.0.1:45680 - "POST /node/69f2a8c4e2fa3d7693ba6e85 HTTP/1.0" 202 Accepted 2026-05-02 00:56:53,462 - lava_callback - INFO - Starting async job submit for node 69f2a8c4e2fa3d7693ba6e85 2026-05-02 00:56:53,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:56:53,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8c4e2fa3d7693ba6e85 HTTP/1.1" 200 None 2026-05-02 00:56:53,490 - lava_callback - DEBUG - Processing node 69f2a8c4e2fa3d7693ba6e85: baseline-arm64-clabbe 2026-05-02 00:56:53,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8c4e2fa3d7693ba6e85 2026-05-02 00:56:53,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:56:53,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8c4e2fa3d7693ba6e85 HTTP/1.1" 200 None 2026-05-02 00:56:53,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:56:53,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 00:56:53,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:56:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8c4e2fa3d7693ba6e85 HTTP/1.1" 200 None 2026-05-02 00:56:53,586 - lava_callback - ERROR - Error processing callback for node 69f2a8c4e2fa3d7693ba6e85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46554 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:57:11,429 - lava_callback - INFO - Received callback for node 69f2a8d9e2fa3d7693ba6ee1 INFO: 172.20.0.1:59688 - "POST /node/69f2a8d9e2fa3d7693ba6ee1 HTTP/1.0" 202 Accepted 2026-05-02 00:57:11,438 - lava_callback - INFO - Starting async job submit for node 69f2a8d9e2fa3d7693ba6ee1 2026-05-02 00:57:11,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:11,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8d9e2fa3d7693ba6ee1 HTTP/1.1" 200 None 2026-05-02 00:57:11,484 - lava_callback - DEBUG - Processing node 69f2a8d9e2fa3d7693ba6ee1: baseline-arm64-baylibre 2026-05-02 00:57:11,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8d9e2fa3d7693ba6ee1 2026-05-02 00:57:11,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:11,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8d9e2fa3d7693ba6ee1 HTTP/1.1" 200 None 2026-05-02 00:57:11,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:11,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:11,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:11,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8d9e2fa3d7693ba6ee1 HTTP/1.1" 200 None 2026-05-02 00:57:11,572 - lava_callback - ERROR - Error processing callback for node 69f2a8d9e2fa3d7693ba6ee1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:12,082 - lava_callback - INFO - Received callback for node 69f2a8dae2fa3d7693ba6ee4 INFO: 172.20.0.1:59698 - "POST /node/69f2a8dae2fa3d7693ba6ee4 HTTP/1.0" 202 Accepted 2026-05-02 00:57:12,092 - lava_callback - INFO - Starting async job submit for node 69f2a8dae2fa3d7693ba6ee4 2026-05-02 00:57:12,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dae2fa3d7693ba6ee4 HTTP/1.1" 200 None 2026-05-02 00:57:12,117 - lava_callback - DEBUG - Processing node 69f2a8dae2fa3d7693ba6ee4: baseline-arm64-baylibre 2026-05-02 00:57:12,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8dae2fa3d7693ba6ee4 2026-05-02 00:57:12,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dae2fa3d7693ba6ee4 HTTP/1.1" 200 None 2026-05-02 00:57:12,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:12,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8dae2fa3d7693ba6ee4 HTTP/1.1" 200 None 2026-05-02 00:57:12,211 - lava_callback - ERROR - Error processing callback for node 69f2a8dae2fa3d7693ba6ee4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:12,777 - lava_callback - INFO - Received callback for node 69f2a8dbe2fa3d7693ba6ee7 INFO: 172.20.0.1:59704 - "POST /node/69f2a8dbe2fa3d7693ba6ee7 HTTP/1.0" 202 Accepted 2026-05-02 00:57:12,790 - lava_callback - INFO - Starting async job submit for node 69f2a8dbe2fa3d7693ba6ee7 2026-05-02 00:57:12,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dbe2fa3d7693ba6ee7 HTTP/1.1" 200 None 2026-05-02 00:57:12,864 - lava_callback - DEBUG - Processing node 69f2a8dbe2fa3d7693ba6ee7: baseline-arm64-baylibre 2026-05-02 00:57:12,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8dbe2fa3d7693ba6ee7 2026-05-02 00:57:12,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dbe2fa3d7693ba6ee7 HTTP/1.1" 200 None 2026-05-02 00:57:12,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:12,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:12,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8dbe2fa3d7693ba6ee7 HTTP/1.1" 200 None 2026-05-02 00:57:12,964 - lava_callback - ERROR - Error processing callback for node 69f2a8dbe2fa3d7693ba6ee7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:13,445 - lava_callback - INFO - Received callback for node 69f2a8dde2fa3d7693ba6eee INFO: 172.20.0.1:59714 - "POST /node/69f2a8dde2fa3d7693ba6eee HTTP/1.0" 202 Accepted 2026-05-02 00:57:13,456 - lava_callback - INFO - Starting async job submit for node 69f2a8dde2fa3d7693ba6eee 2026-05-02 00:57:13,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:13,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dde2fa3d7693ba6eee HTTP/1.1" 200 None 2026-05-02 00:57:13,492 - lava_callback - DEBUG - Processing node 69f2a8dde2fa3d7693ba6eee: baseline-arm64-baylibre 2026-05-02 00:57:13,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8dde2fa3d7693ba6eee 2026-05-02 00:57:13,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:13,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dde2fa3d7693ba6eee HTTP/1.1" 200 None 2026-05-02 00:57:13,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:13,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:13,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:13,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8dde2fa3d7693ba6eee HTTP/1.1" 200 None 2026-05-02 00:57:13,589 - lava_callback - ERROR - Error processing callback for node 69f2a8dde2fa3d7693ba6eee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:14,122 - lava_callback - INFO - Received callback for node 69f2a8dfe2fa3d7693ba6ef1 INFO: 172.20.0.1:59728 - "POST /node/69f2a8dfe2fa3d7693ba6ef1 HTTP/1.0" 202 Accepted 2026-05-02 00:57:14,134 - lava_callback - INFO - Starting async job submit for node 69f2a8dfe2fa3d7693ba6ef1 2026-05-02 00:57:14,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dfe2fa3d7693ba6ef1 HTTP/1.1" 200 None 2026-05-02 00:57:14,161 - lava_callback - DEBUG - Processing node 69f2a8dfe2fa3d7693ba6ef1: baseline-arm64-baylibre 2026-05-02 00:57:14,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8dfe2fa3d7693ba6ef1 2026-05-02 00:57:14,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8dfe2fa3d7693ba6ef1 HTTP/1.1" 200 None 2026-05-02 00:57:14,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:14,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8dfe2fa3d7693ba6ef1 HTTP/1.1" 200 None 2026-05-02 00:57:14,230 - lava_callback - ERROR - Error processing callback for node 69f2a8dfe2fa3d7693ba6ef1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:14,816 - lava_callback - INFO - Received callback for node 69f2a8e0e2fa3d7693ba6ef4 INFO: 172.20.0.1:59740 - "POST /node/69f2a8e0e2fa3d7693ba6ef4 HTTP/1.0" 202 Accepted 2026-05-02 00:57:14,838 - lava_callback - INFO - Starting async job submit for node 69f2a8e0e2fa3d7693ba6ef4 2026-05-02 00:57:14,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8e0e2fa3d7693ba6ef4 HTTP/1.1" 200 None 2026-05-02 00:57:14,864 - lava_callback - DEBUG - Processing node 69f2a8e0e2fa3d7693ba6ef4: baseline-arm64-baylibre 2026-05-02 00:57:14,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8e0e2fa3d7693ba6ef4 2026-05-02 00:57:14,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8e0e2fa3d7693ba6ef4 HTTP/1.1" 200 None 2026-05-02 00:57:14,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:14,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:14,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8e0e2fa3d7693ba6ef4 HTTP/1.1" 200 None 2026-05-02 00:57:14,974 - lava_callback - ERROR - Error processing callback for node 69f2a8e0e2fa3d7693ba6ef4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37538 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:57:33,448 - lava_callback - INFO - Received callback for node 69f2a8efe2fa3d7693ba6f42 INFO: 172.20.0.1:41880 - "POST /node/69f2a8efe2fa3d7693ba6f42 HTTP/1.0" 202 Accepted 2026-05-02 00:57:33,457 - lava_callback - INFO - Starting async job submit for node 69f2a8efe2fa3d7693ba6f42 2026-05-02 00:57:33,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:33,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8efe2fa3d7693ba6f42 HTTP/1.1" 200 None 2026-05-02 00:57:33,485 - lava_callback - DEBUG - Processing node 69f2a8efe2fa3d7693ba6f42: baseline-arm64-clabbe 2026-05-02 00:57:33,485 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8efe2fa3d7693ba6f42 2026-05-02 00:57:33,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:33,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8efe2fa3d7693ba6f42 HTTP/1.1" 200 None 2026-05-02 00:57:33,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:33,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:33,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:33,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8efe2fa3d7693ba6f42 HTTP/1.1" 200 None 2026-05-02 00:57:33,540 - lava_callback - ERROR - Error processing callback for node 69f2a8efe2fa3d7693ba6f42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:34,142 - lava_callback - INFO - Received callback for node 69f2a8f1e2fa3d7693ba6f45 INFO: 172.20.0.1:41892 - "POST /node/69f2a8f1e2fa3d7693ba6f45 HTTP/1.0" 202 Accepted 2026-05-02 00:57:34,148 - lava_callback - INFO - Starting async job submit for node 69f2a8f1e2fa3d7693ba6f45 2026-05-02 00:57:34,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f1e2fa3d7693ba6f45 HTTP/1.1" 200 None 2026-05-02 00:57:34,175 - lava_callback - DEBUG - Processing node 69f2a8f1e2fa3d7693ba6f45: baseline-arm64-clabbe 2026-05-02 00:57:34,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8f1e2fa3d7693ba6f45 2026-05-02 00:57:34,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f1e2fa3d7693ba6f45 HTTP/1.1" 200 None 2026-05-02 00:57:34,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:34,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8f1e2fa3d7693ba6f45 HTTP/1.1" 200 None 2026-05-02 00:57:34,220 - lava_callback - ERROR - Error processing callback for node 69f2a8f1e2fa3d7693ba6f45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:34,837 - lava_callback - INFO - Received callback for node 69f2a8f1e2fa3d7693ba6f48 INFO: 172.20.0.1:41906 - "POST /node/69f2a8f1e2fa3d7693ba6f48 HTTP/1.0" 202 Accepted 2026-05-02 00:57:34,843 - lava_callback - INFO - Starting async job submit for node 69f2a8f1e2fa3d7693ba6f48 2026-05-02 00:57:34,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f1e2fa3d7693ba6f48 HTTP/1.1" 200 None 2026-05-02 00:57:34,876 - lava_callback - DEBUG - Processing node 69f2a8f1e2fa3d7693ba6f48: baseline-arm64-clabbe 2026-05-02 00:57:34,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8f1e2fa3d7693ba6f48 2026-05-02 00:57:34,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f1e2fa3d7693ba6f48 HTTP/1.1" 200 None 2026-05-02 00:57:34,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:34,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:34,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8f1e2fa3d7693ba6f48 HTTP/1.1" 200 None 2026-05-02 00:57:34,985 - lava_callback - ERROR - Error processing callback for node 69f2a8f1e2fa3d7693ba6f48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:35,550 - lava_callback - INFO - Received callback for node 69f2a8f2e2fa3d7693ba6f4b INFO: 172.20.0.1:41914 - "POST /node/69f2a8f2e2fa3d7693ba6f4b HTTP/1.0" 202 Accepted 2026-05-02 00:57:35,556 - lava_callback - INFO - Starting async job submit for node 69f2a8f2e2fa3d7693ba6f4b 2026-05-02 00:57:35,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:35,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f2e2fa3d7693ba6f4b HTTP/1.1" 200 None 2026-05-02 00:57:35,570 - lava_callback - DEBUG - Processing node 69f2a8f2e2fa3d7693ba6f4b: baseline-arm64-clabbe 2026-05-02 00:57:35,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8f2e2fa3d7693ba6f4b 2026-05-02 00:57:35,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:35,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f2e2fa3d7693ba6f4b HTTP/1.1" 200 None 2026-05-02 00:57:35,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:35,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:35,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:35,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8f2e2fa3d7693ba6f4b HTTP/1.1" 200 None 2026-05-02 00:57:35,665 - lava_callback - ERROR - Error processing callback for node 69f2a8f2e2fa3d7693ba6f4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:36,239 - lava_callback - INFO - Received callback for node 69f2a8f3e2fa3d7693ba6f4e INFO: 172.20.0.1:41924 - "POST /node/69f2a8f3e2fa3d7693ba6f4e HTTP/1.0" 202 Accepted 2026-05-02 00:57:36,245 - lava_callback - INFO - Starting async job submit for node 69f2a8f3e2fa3d7693ba6f4e 2026-05-02 00:57:36,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:36,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f3e2fa3d7693ba6f4e HTTP/1.1" 200 None 2026-05-02 00:57:36,270 - lava_callback - DEBUG - Processing node 69f2a8f3e2fa3d7693ba6f4e: baseline-arm64-clabbe 2026-05-02 00:57:36,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8f3e2fa3d7693ba6f4e 2026-05-02 00:57:36,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:36,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f3e2fa3d7693ba6f4e HTTP/1.1" 200 None 2026-05-02 00:57:36,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:36,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:36,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:36,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8f3e2fa3d7693ba6f4e HTTP/1.1" 200 None 2026-05-02 00:57:36,317 - lava_callback - ERROR - Error processing callback for node 69f2a8f3e2fa3d7693ba6f4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41926 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:57:36,934 - lava_callback - INFO - Received callback for node 69f2a8f4e2fa3d7693ba6f56 INFO: 172.20.0.1:41938 - "POST /node/69f2a8f4e2fa3d7693ba6f56 HTTP/1.0" 202 Accepted 2026-05-02 00:57:36,944 - lava_callback - INFO - Starting async job submit for node 69f2a8f4e2fa3d7693ba6f56 2026-05-02 00:57:36,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:36,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f4e2fa3d7693ba6f56 HTTP/1.1" 200 None 2026-05-02 00:57:36,960 - lava_callback - DEBUG - Processing node 69f2a8f4e2fa3d7693ba6f56: baseline-arm64-clabbe 2026-05-02 00:57:36,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8f4e2fa3d7693ba6f56 2026-05-02 00:57:36,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:36,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f4e2fa3d7693ba6f56 HTTP/1.1" 200 None 2026-05-02 00:57:36,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:36,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:36,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:37,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8f4e2fa3d7693ba6f56 HTTP/1.1" 200 None 2026-05-02 00:57:37,011 - lava_callback - ERROR - Error processing callback for node 69f2a8f4e2fa3d7693ba6f56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:37,626 - lava_callback - INFO - Received callback for node 69f2a8f4e2fa3d7693ba6f53 INFO: 172.20.0.1:41940 - "POST /node/69f2a8f4e2fa3d7693ba6f53 HTTP/1.0" 202 Accepted 2026-05-02 00:57:37,632 - lava_callback - INFO - Starting async job submit for node 69f2a8f4e2fa3d7693ba6f53 2026-05-02 00:57:37,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:37,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f4e2fa3d7693ba6f53 HTTP/1.1" 200 None 2026-05-02 00:57:37,647 - lava_callback - DEBUG - Processing node 69f2a8f4e2fa3d7693ba6f53: baseline-arm64-clabbe 2026-05-02 00:57:37,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a8f4e2fa3d7693ba6f53 2026-05-02 00:57:37,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:37,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a8f4e2fa3d7693ba6f53 HTTP/1.1" 200 None 2026-05-02 00:57:37,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:37,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fd4e2fa3d7693ba43c1 HTTP/1.1" 200 None 2026-05-02 00:57:37,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:37,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a8f4e2fa3d7693ba6f53 HTTP/1.1" 200 None 2026-05-02 00:57:37,693 - lava_callback - ERROR - Error processing callback for node 69f2a8f4e2fa3d7693ba6f53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 00:57:45,764 - lava_callback - INFO - Received callback for node 69f54bdeed4ed40b022406dd INFO: 172.20.0.1:49830 - "POST /node/69f54bdeed4ed40b022406dd HTTP/1.0" 202 Accepted 2026-05-02 00:57:45,774 - lava_callback - INFO - Starting async job submit for node 69f54bdeed4ed40b022406dd 2026-05-02 00:57:45,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:45,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bdeed4ed40b022406dd HTTP/1.1" 200 None 2026-05-02 00:57:45,835 - lava_callback - DEBUG - Processing node 69f54bdeed4ed40b022406dd: baseline-arm64 2026-05-02 00:57:46,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:46,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:46,158 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bdeed4ed40b022406dd/log.txt.gz 2026-05-02 00:57:46,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:46,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:46,367 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bdeed4ed40b022406dd/lava-logs.yaml 2026-05-02 00:57:46,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:46,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:46,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bdeed4ed40b022406dd/lava_callback.json.gz 2026-05-02 00:57:46,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:46,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bdeed4ed40b022406dd HTTP/1.1" 200 None 2026-05-02 00:57:46,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:46,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:57:46,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:46,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bdeed4ed40b022406dd HTTP/1.1" 200 None 2026-05-02 00:57:46,541 - lava_callback - INFO - Completed processing callback for node 69f54bdeed4ed40b022406dd INFO: 172.20.0.1:49840 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:57:46,843 - lava_callback - INFO - Received callback for node 69f54bebed4ed40b0224072b INFO: 172.20.0.1:49846 - "POST /node/69f54bebed4ed40b0224072b HTTP/1.0" 202 Accepted 2026-05-02 00:57:46,849 - lava_callback - INFO - Starting async job submit for node 69f54bebed4ed40b0224072b 2026-05-02 00:57:46,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:46,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bebed4ed40b0224072b HTTP/1.1" 200 None 2026-05-02 00:57:46,883 - lava_callback - DEBUG - Processing node 69f54bebed4ed40b0224072b: baseline-arm64 2026-05-02 00:57:47,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:47,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:47,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bebed4ed40b0224072b/log.txt.gz 2026-05-02 00:57:47,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:47,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:47,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bebed4ed40b0224072b/lava-logs.yaml 2026-05-02 00:57:47,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:47,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:47,441 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bebed4ed40b0224072b/lava_callback.json.gz 2026-05-02 00:57:47,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:47,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bebed4ed40b0224072b HTTP/1.1" 200 None 2026-05-02 00:57:47,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:47,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:57:47,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:47,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bebed4ed40b0224072b HTTP/1.1" 200 None 2026-05-02 00:57:47,590 - lava_callback - INFO - Completed processing callback for node 69f54bebed4ed40b0224072b 2026-05-02 00:57:47,652 - lava_callback - INFO - Received callback for node 69f54bfaed4ed40b02240754 INFO: 172.20.0.1:49856 - "POST /node/69f54bfaed4ed40b02240754 HTTP/1.0" 202 Accepted 2026-05-02 00:57:47,664 - lava_callback - INFO - Starting async job submit for node 69f54bfaed4ed40b02240754 2026-05-02 00:57:47,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:47,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfaed4ed40b02240754 HTTP/1.1" 200 None 2026-05-02 00:57:47,682 - lava_callback - DEBUG - Processing node 69f54bfaed4ed40b02240754: baseline-nfs-arm64 2026-05-02 00:57:47,684 - lava_callback - WARNING - Failed to upload log 2026-05-02 00:57:47,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:47,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:47,776 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfaed4ed40b02240754/lava-logs.yaml 2026-05-02 00:57:47,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:57:47,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:57:47,866 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfaed4ed40b02240754/lava_callback.json.gz 2026-05-02 00:57:47,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:47,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfaed4ed40b02240754 HTTP/1.1" 200 None 2026-05-02 00:57:47,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:47,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:57:47,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:47,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bfaed4ed40b02240754 HTTP/1.1" 200 None 2026-05-02 00:57:47,927 - lava_callback - INFO - Completed processing callback for node 69f54bfaed4ed40b02240754 2026-05-02 00:57:52,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:52,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 00:57:53,520 - lava_callback - INFO - Received callback for node 69f2a904e2fa3d7693ba6f6f INFO: 172.20.0.1:38830 - "POST /node/69f2a904e2fa3d7693ba6f6f HTTP/1.0" 202 Accepted 2026-05-02 00:57:53,526 - lava_callback - INFO - Starting async job submit for node 69f2a904e2fa3d7693ba6f6f 2026-05-02 00:57:53,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:53,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a904e2fa3d7693ba6f6f HTTP/1.1" 200 None 2026-05-02 00:57:53,541 - lava_callback - DEBUG - Processing node 69f2a904e2fa3d7693ba6f6f: baseline-arm-clabbe 2026-05-02 00:57:53,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2a904e2fa3d7693ba6f6f 2026-05-02 00:57:53,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:53,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2a904e2fa3d7693ba6f6f HTTP/1.1" 200 None 2026-05-02 00:57:53,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:53,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 00:57:53,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:57:53,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2a904e2fa3d7693ba6f6f HTTP/1.1" 200 None 2026-05-02 00:57:53,591 - lava_callback - ERROR - Error processing callback for node 69f2a904e2fa3d7693ba6f6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47362 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:58:22,936 - lava_callback - INFO - Received callback for node 69f54be0ed4ed40b022406e3 INFO: 172.20.0.1:47368 - "POST /node/69f54be0ed4ed40b022406e3 HTTP/1.0" 202 Accepted 2026-05-02 00:58:22,942 - lava_callback - INFO - Starting async job submit for node 69f54be0ed4ed40b022406e3 2026-05-02 00:58:22,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:22,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be0ed4ed40b022406e3 HTTP/1.1" 200 None 2026-05-02 00:58:22,979 - lava_callback - DEBUG - Processing node 69f54be0ed4ed40b022406e3: baseline-arm64 2026-05-02 00:58:23,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:23,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:23,579 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be0ed4ed40b022406e3/log.txt.gz 2026-05-02 00:58:23,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:23,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:23,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be0ed4ed40b022406e3/lava-logs.yaml 2026-05-02 00:58:23,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:24,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:24,044 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be0ed4ed40b022406e3/lava_callback.json.gz 2026-05-02 00:58:24,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:24,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be0ed4ed40b022406e3 HTTP/1.1" 200 None 2026-05-02 00:58:24,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:24,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:58:24,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:24,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be0ed4ed40b022406e3 HTTP/1.1" 200 None 2026-05-02 00:58:24,126 - lava_callback - INFO - Completed processing callback for node 69f54be0ed4ed40b022406e3 INFO: 172.20.0.1:47378 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:58:31,858 - lava_callback - INFO - Received callback for node 69f54bdfed4ed40b022406e0 INFO: 172.20.0.1:45620 - "POST /node/69f54bdfed4ed40b022406e0 HTTP/1.0" 202 Accepted 2026-05-02 00:58:31,865 - lava_callback - INFO - Starting async job submit for node 69f54bdfed4ed40b022406e0 2026-05-02 00:58:31,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:31,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bdfed4ed40b022406e0 HTTP/1.1" 200 None 2026-05-02 00:58:31,901 - lava_callback - DEBUG - Processing node 69f54bdfed4ed40b022406e0: baseline-arm64 2026-05-02 00:58:32,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:32,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:32,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bdfed4ed40b022406e0/log.txt.gz 2026-05-02 00:58:32,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:32,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:32,513 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bdfed4ed40b022406e0/lava-logs.yaml 2026-05-02 00:58:32,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:32,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:32,625 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bdfed4ed40b022406e0/lava_callback.json.gz 2026-05-02 00:58:32,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:32,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bdfed4ed40b022406e0 HTTP/1.1" 200 None 2026-05-02 00:58:32,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:32,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:58:32,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:32,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bdfed4ed40b022406e0 HTTP/1.1" 200 None 2026-05-02 00:58:32,727 - lava_callback - INFO - Completed processing callback for node 69f54bdfed4ed40b022406e0 INFO: 172.20.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:58:52,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:52,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 00:58:55,848 - lava_callback - INFO - Received callback for node 69f54c21ed4ed40b022407fb INFO: 172.20.0.1:45772 - "POST /node/69f54c21ed4ed40b022407fb HTTP/1.0" 202 Accepted 2026-05-02 00:58:55,855 - lava_callback - INFO - Starting async job submit for node 69f54c21ed4ed40b022407fb 2026-05-02 00:58:55,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:55,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c21ed4ed40b022407fb HTTP/1.1" 200 None 2026-05-02 00:58:55,909 - lava_callback - DEBUG - Processing node 69f54c21ed4ed40b022407fb: baseline-arm64-pengutronix 2026-05-02 00:58:56,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:56,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:56,273 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c21ed4ed40b022407fb/log.txt.gz 2026-05-02 00:58:56,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:58:56,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:56,531 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c21ed4ed40b022407fb/lava-logs.yaml 2026-05-02 00:58:56,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45778 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:58:56,653 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:58:56,655 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c21ed4ed40b022407fb/lava_callback.json.gz 2026-05-02 00:58:56,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:56,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c21ed4ed40b022407fb HTTP/1.1" 200 None 2026-05-02 00:58:56,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:56,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:58:56,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:58:56,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c21ed4ed40b022407fb HTTP/1.1" 200 None 2026-05-02 00:58:56,752 - lava_callback - INFO - Completed processing callback for node 69f54c21ed4ed40b022407fb 2026-05-02 00:59:03,203 - lava_callback - INFO - Received callback for node 69f54c22ed4ed40b022407fe INFO: 172.20.0.1:46664 - "POST /node/69f54c22ed4ed40b022407fe HTTP/1.0" 202 Accepted 2026-05-02 00:59:03,211 - lava_callback - INFO - Starting async job submit for node 69f54c22ed4ed40b022407fe 2026-05-02 00:59:03,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:03,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c22ed4ed40b022407fe HTTP/1.1" 200 None 2026-05-02 00:59:03,261 - lava_callback - DEBUG - Processing node 69f54c22ed4ed40b022407fe: baseline-arm64-pengutronix 2026-05-02 00:59:03,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:03,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:03,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c22ed4ed40b022407fe/log.txt.gz 2026-05-02 00:59:03,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:03,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:03,914 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c22ed4ed40b022407fe/lava-logs.yaml 2026-05-02 00:59:03,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:04,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:04,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c22ed4ed40b022407fe/lava_callback.json.gz 2026-05-02 00:59:04,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:04,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c22ed4ed40b022407fe HTTP/1.1" 200 None 2026-05-02 00:59:04,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:04,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:59:04,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:04,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c22ed4ed40b022407fe HTTP/1.1" 200 None 2026-05-02 00:59:04,111 - lava_callback - INFO - Completed processing callback for node 69f54c22ed4ed40b022407fe INFO: 172.20.0.1:46680 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:59:06,937 - lava_callback - INFO - Received callback for node 69f54c24ed4ed40b02240832 INFO: 172.20.0.1:46692 - "POST /node/69f54c24ed4ed40b02240832 HTTP/1.0" 202 Accepted 2026-05-02 00:59:06,945 - lava_callback - INFO - Starting async job submit for node 69f54c24ed4ed40b02240832 2026-05-02 00:59:06,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:07,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c24ed4ed40b02240832 HTTP/1.1" 200 None 2026-05-02 00:59:07,002 - lava_callback - DEBUG - Processing node 69f54c24ed4ed40b02240832: baseline-arm64-foundriesio 2026-05-02 00:59:07,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:07,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:07,451 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f54c24ed4ed40b02240832/log.txt.gz 2026-05-02 00:59:07,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:07,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:07,873 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f54c24ed4ed40b02240832/lava-logs.yaml 2026-05-02 00:59:07,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:08,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:08,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f54c24ed4ed40b02240832/lava_callback.json.gz 2026-05-02 00:59:08,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:08,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c24ed4ed40b02240832 HTTP/1.1" 200 None 2026-05-02 00:59:08,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:08,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:59:08,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:08,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c24ed4ed40b02240832 HTTP/1.1" 200 None 2026-05-02 00:59:08,161 - lava_callback - INFO - Completed processing callback for node 69f54c24ed4ed40b02240832 2026-05-02 00:59:12,747 - lava_callback - INFO - Received callback for node 69f54c23ed4ed40b0224082f INFO: 172.20.0.1:49546 - "POST /node/69f54c23ed4ed40b0224082f HTTP/1.0" 202 Accepted 2026-05-02 00:59:12,753 - lava_callback - INFO - Starting async job submit for node 69f54c23ed4ed40b0224082f 2026-05-02 00:59:12,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:12,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c23ed4ed40b0224082f HTTP/1.1" 200 None 2026-05-02 00:59:12,794 - lava_callback - DEBUG - Processing node 69f54c23ed4ed40b0224082f: baseline-arm64-pengutronix 2026-05-02 00:59:13,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:13,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:13,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c23ed4ed40b0224082f/log.txt.gz 2026-05-02 00:59:13,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:13,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:13,640 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c23ed4ed40b0224082f/lava-logs.yaml 2026-05-02 00:59:13,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:13,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:13,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f54c23ed4ed40b0224082f/lava_callback.json.gz 2026-05-02 00:59:13,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:13,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c23ed4ed40b0224082f HTTP/1.1" 200 None 2026-05-02 00:59:13,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:13,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:59:13,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:13,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c23ed4ed40b0224082f HTTP/1.1" 200 None 2026-05-02 00:59:13,921 - lava_callback - INFO - Completed processing callback for node 69f54c23ed4ed40b0224082f 2026-05-02 00:59:15,013 - lava_callback - INFO - Received callback for node 69f54c20ed4ed40b022407f8 INFO: 172.20.0.1:49558 - "POST /node/69f54c20ed4ed40b022407f8 HTTP/1.0" 202 Accepted 2026-05-02 00:59:15,033 - lava_callback - INFO - Starting async job submit for node 69f54c20ed4ed40b022407f8 2026-05-02 00:59:15,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:15,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c20ed4ed40b022407f8 HTTP/1.1" 200 None 2026-05-02 00:59:15,152 - lava_callback - DEBUG - Processing node 69f54c20ed4ed40b022407f8: baseline-arm64-kontron INFO: 172.20.0.1:49570 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 00:59:15,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:15,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:15,764 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69f54c20ed4ed40b022407f8/log.txt.gz 2026-05-02 00:59:16,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:16,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:16,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69f54c20ed4ed40b022407f8/lava-logs.yaml 2026-05-02 00:59:16,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:16,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:16,524 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69f54c20ed4ed40b022407f8/lava_callback.json.gz 2026-05-02 00:59:16,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:16,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c20ed4ed40b022407f8 HTTP/1.1" 200 None 2026-05-02 00:59:16,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:49586 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:59:16,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:59:16,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:16,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c20ed4ed40b022407f8 HTTP/1.1" 200 None 2026-05-02 00:59:16,616 - lava_callback - INFO - Completed processing callback for node 69f54c20ed4ed40b022407f8 INFO: 172.20.0.1:42912 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:59:22,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:22,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:42926 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:59:34,566 - lava_callback - INFO - Received callback for node 69f54c25ed4ed40b02240835 INFO: 172.20.0.1:55126 - "POST /node/69f54c25ed4ed40b02240835 HTTP/1.0" 202 Accepted 2026-05-02 00:59:34,581 - lava_callback - INFO - Starting async job submit for node 69f54c25ed4ed40b02240835 2026-05-02 00:59:34,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:34,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c25ed4ed40b02240835 HTTP/1.1" 200 None 2026-05-02 00:59:34,667 - lava_callback - DEBUG - Processing node 69f54c25ed4ed40b02240835: baseline-arm64-foundriesio 2026-05-02 00:59:35,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:35,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:35,479 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f54c25ed4ed40b02240835/log.txt.gz 2026-05-02 00:59:35,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:36,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:36,153 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f54c25ed4ed40b02240835/lava-logs.yaml 2026-05-02 00:59:36,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 00:59:36,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 00:59:36,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f54c25ed4ed40b02240835/lava_callback.json.gz 2026-05-02 00:59:36,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:36,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c25ed4ed40b02240835 HTTP/1.1" 200 None 2026-05-02 00:59:36,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:36,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 00:59:36,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:36,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c25ed4ed40b02240835 HTTP/1.1" 200 None 2026-05-02 00:59:36,405 - lava_callback - INFO - Completed processing callback for node 69f54c25ed4ed40b02240835 INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK 2026-05-02 00:59:52,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 00:59:52,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56790 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:00:04,314 - lava_callback - INFO - Received callback for node 69f54be2ed4ed40b022406e9 INFO: 172.20.0.1:49822 - "POST /node/69f54be2ed4ed40b022406e9 HTTP/1.0" 202 Accepted 2026-05-02 01:00:04,329 - lava_callback - INFO - Starting async job submit for node 69f54be2ed4ed40b022406e9 2026-05-02 01:00:04,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:04,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be2ed4ed40b022406e9 HTTP/1.1" 200 None 2026-05-02 01:00:04,433 - lava_callback - DEBUG - Processing node 69f54be2ed4ed40b022406e9: baseline-arm64 2026-05-02 01:00:04,694 - lava_callback - WARNING - Failed to upload log 2026-05-02 01:00:04,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:04,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:04,945 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be2ed4ed40b022406e9/lava-logs.yaml 2026-05-02 01:00:04,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:05,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:05,064 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be2ed4ed40b022406e9/lava_callback.json.gz 2026-05-02 01:00:05,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:05,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be2ed4ed40b022406e9 HTTP/1.1" 200 None 2026-05-02 01:00:05,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:05,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:05,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:05,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be2ed4ed40b022406e9 HTTP/1.1" 200 None 2026-05-02 01:00:05,241 - lava_callback - INFO - Completed processing callback for node 69f54be2ed4ed40b022406e9 2026-05-02 01:00:05,352 - lava_callback - INFO - Received callback for node 69f54bf0ed4ed40b02240739 INFO: 172.20.0.1:49834 - "POST /node/69f54bf0ed4ed40b02240739 HTTP/1.0" 202 Accepted 2026-05-02 01:00:05,363 - lava_callback - INFO - Starting async job submit for node 69f54bf0ed4ed40b02240739 2026-05-02 01:00:05,382 - lava_callback - INFO - Received callback for node 69f54be7ed4ed40b02240718 INFO: 172.20.0.1:49846 - "POST /node/69f54be7ed4ed40b02240718 HTTP/1.0" 202 Accepted 2026-05-02 01:00:05,402 - lava_callback - INFO - Starting async job submit for node 69f54be7ed4ed40b02240718 2026-05-02 01:00:05,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:05,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:05,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf0ed4ed40b02240739 HTTP/1.1" 200 None 2026-05-02 01:00:05,522 - lava_callback - DEBUG - Processing node 69f54bf0ed4ed40b02240739: baseline-arm64 2026-05-02 01:00:05,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be7ed4ed40b02240718 HTTP/1.1" 200 None 2026-05-02 01:00:05,530 - lava_callback - DEBUG - Processing node 69f54be7ed4ed40b02240718: baseline-arm64 2026-05-02 01:00:05,837 - lava_callback - INFO - Received callback for node 69f54be4ed4ed40b022406ec 2026-05-02 01:00:05,891 - lava_callback - INFO - Starting async job submit for node 69f54be4ed4ed40b022406ec INFO: 172.20.0.1:49862 - "POST /node/69f54be4ed4ed40b022406ec HTTP/1.0" 202 Accepted 2026-05-02 01:00:06,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:06,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be4ed4ed40b022406ec HTTP/1.1" 200 None 2026-05-02 01:00:06,284 - lava_callback - DEBUG - Processing node 69f54be4ed4ed40b022406ec: baseline-arm64 2026-05-02 01:00:06,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49878 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:00:06,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:06,671 - lava_callback - WARNING - Failed to upload log 2026-05-02 01:00:06,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:06,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf0ed4ed40b02240739/log.txt.gz 2026-05-02 01:00:06,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:06,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:06,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be7ed4ed40b02240718/log.txt.gz 2026-05-02 01:00:06,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:07,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be4ed4ed40b022406ec/lava-logs.yaml 2026-05-02 01:00:07,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:07,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:07,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:07,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:07,235 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be4ed4ed40b022406ec/lava_callback.json.gz 2026-05-02 01:00:07,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:07,335 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf0ed4ed40b02240739/lava-logs.yaml 2026-05-02 01:00:07,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:07,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be4ed4ed40b022406ec HTTP/1.1" 200 None 2026-05-02 01:00:07,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:07,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be4ed4ed40b022406ec HTTP/1.1" 200 None 2026-05-02 01:00:07,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:07,454 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be7ed4ed40b02240718/lava-logs.yaml 2026-05-02 01:00:07,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:07,516 - lava_callback - INFO - Completed processing callback for node 69f54be4ed4ed40b022406ec 2026-05-02 01:00:07,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:07,519 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf0ed4ed40b02240739/lava_callback.json.gz 2026-05-02 01:00:07,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf0ed4ed40b02240739 HTTP/1.1" 200 None 2026-05-02 01:00:07,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:07,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf0ed4ed40b02240739 HTTP/1.1" 200 None 2026-05-02 01:00:07,659 - lava_callback - INFO - Completed processing callback for node 69f54bf0ed4ed40b02240739 2026-05-02 01:00:07,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:07,667 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be7ed4ed40b02240718/lava_callback.json.gz 2026-05-02 01:00:07,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be7ed4ed40b02240718 HTTP/1.1" 200 None 2026-05-02 01:00:07,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:07,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:07,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be7ed4ed40b02240718 HTTP/1.1" 200 None 2026-05-02 01:00:07,844 - lava_callback - INFO - Completed processing callback for node 69f54be7ed4ed40b02240718 2026-05-02 01:00:09,594 - lava_callback - INFO - Received callback for node 69f54be9ed4ed40b0224071f INFO: 172.20.0.1:49882 - "POST /node/69f54be9ed4ed40b0224071f HTTP/1.0" 202 Accepted 2026-05-02 01:00:09,601 - lava_callback - INFO - Starting async job submit for node 69f54be9ed4ed40b0224071f 2026-05-02 01:00:09,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:09,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be9ed4ed40b0224071f HTTP/1.1" 200 None 2026-05-02 01:00:09,663 - lava_callback - DEBUG - Processing node 69f54be9ed4ed40b0224071f: baseline-arm64 2026-05-02 01:00:10,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:10,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:10,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be9ed4ed40b0224071f/log.txt.gz 2026-05-02 01:00:10,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:10,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:10,690 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be9ed4ed40b0224071f/lava-logs.yaml 2026-05-02 01:00:10,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:10,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:10,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be9ed4ed40b0224071f/lava_callback.json.gz 2026-05-02 01:00:10,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:10,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be9ed4ed40b0224071f HTTP/1.1" 200 None 2026-05-02 01:00:10,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:10,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:10,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:10,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be9ed4ed40b0224071f HTTP/1.1" 200 None 2026-05-02 01:00:10,887 - lava_callback - INFO - Completed processing callback for node 69f54be9ed4ed40b0224071f 2026-05-02 01:00:13,294 - lava_callback - INFO - Received callback for node 69f54bf1ed4ed40b0224073c INFO: 172.20.0.1:37876 - "POST /node/69f54bf1ed4ed40b0224073c HTTP/1.0" 202 Accepted 2026-05-02 01:00:13,304 - lava_callback - INFO - Starting async job submit for node 69f54bf1ed4ed40b0224073c 2026-05-02 01:00:13,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:13,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf1ed4ed40b0224073c HTTP/1.1" 200 None 2026-05-02 01:00:13,356 - lava_callback - DEBUG - Processing node 69f54bf1ed4ed40b0224073c: baseline-arm64 2026-05-02 01:00:13,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:13,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:13,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf1ed4ed40b0224073c/log.txt.gz 2026-05-02 01:00:14,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:14,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:14,249 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf1ed4ed40b0224073c/lava-logs.yaml 2026-05-02 01:00:14,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:14,263 - lava_callback - INFO - Received callback for node 69f54beced4ed40b0224072e INFO: 172.20.0.1:37880 - "POST /node/69f54beced4ed40b0224072e HTTP/1.0" 202 Accepted 2026-05-02 01:00:14,272 - lava_callback - INFO - Starting async job submit for node 69f54beced4ed40b0224072e 2026-05-02 01:00:14,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:14,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54beced4ed40b0224072e HTTP/1.1" 200 None 2026-05-02 01:00:14,317 - lava_callback - DEBUG - Processing node 69f54beced4ed40b0224072e: baseline-arm64 2026-05-02 01:00:14,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:14,477 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf1ed4ed40b0224073c/lava_callback.json.gz 2026-05-02 01:00:14,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:14,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:14,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf1ed4ed40b0224073c HTTP/1.1" 200 None 2026-05-02 01:00:14,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:14,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:14,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:14,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf1ed4ed40b0224073c HTTP/1.1" 200 None 2026-05-02 01:00:14,631 - lava_callback - INFO - Completed processing callback for node 69f54bf1ed4ed40b0224073c 2026-05-02 01:00:14,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:14,695 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beced4ed40b0224072e/log.txt.gz 2026-05-02 01:00:14,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:15,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:15,021 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beced4ed40b0224072e/lava-logs.yaml 2026-05-02 01:00:15,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:15,076 - lava_callback - INFO - Received callback for node 69f54beaed4ed40b02240726 INFO: 172.20.0.1:37892 - "POST /node/69f54beaed4ed40b02240726 HTTP/1.0" 202 Accepted 2026-05-02 01:00:15,084 - lava_callback - INFO - Starting async job submit for node 69f54beaed4ed40b02240726 2026-05-02 01:00:15,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:15,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:15,124 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beced4ed40b0224072e/lava_callback.json.gz 2026-05-02 01:00:15,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54beaed4ed40b02240726 HTTP/1.1" 200 None 2026-05-02 01:00:15,160 - lava_callback - DEBUG - Processing node 69f54beaed4ed40b02240726: baseline-arm64 2026-05-02 01:00:15,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:37900 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:00:15,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54beced4ed40b0224072e HTTP/1.1" 200 None 2026-05-02 01:00:15,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:15,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:15,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:15,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54beced4ed40b0224072e HTTP/1.1" 200 None 2026-05-02 01:00:15,537 - lava_callback - INFO - Completed processing callback for node 69f54beced4ed40b0224072e 2026-05-02 01:00:15,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:15,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:15,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beaed4ed40b02240726/log.txt.gz 2026-05-02 01:00:16,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:16,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:16,256 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beaed4ed40b02240726/lava-logs.yaml 2026-05-02 01:00:16,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:16,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:16,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beaed4ed40b02240726/lava_callback.json.gz 2026-05-02 01:00:16,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:16,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54beaed4ed40b02240726 HTTP/1.1" 200 None 2026-05-02 01:00:16,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:16,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:16,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:16,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54beaed4ed40b02240726 HTTP/1.1" 200 None 2026-05-02 01:00:16,468 - lava_callback - INFO - Completed processing callback for node 69f54beaed4ed40b02240726 INFO: 172.20.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53512 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:00:21,805 - lava_callback - INFO - Received callback for node 69f54be6ed4ed40b022406f2 INFO: 172.20.0.1:53524 - "POST /node/69f54be6ed4ed40b022406f2 HTTP/1.0" 202 Accepted 2026-05-02 01:00:21,816 - lava_callback - INFO - Starting async job submit for node 69f54be6ed4ed40b022406f2 2026-05-02 01:00:21,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:21,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be6ed4ed40b022406f2 HTTP/1.1" 200 None 2026-05-02 01:00:21,863 - lava_callback - DEBUG - Processing node 69f54be6ed4ed40b022406f2: baseline-arm64 2026-05-02 01:00:22,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:22,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:22,284 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be6ed4ed40b022406f2/log.txt.gz 2026-05-02 01:00:22,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:22,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:22,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:22,578 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be6ed4ed40b022406f2/lava-logs.yaml 2026-05-02 01:00:22,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:22,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-05-02 01:00:22,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:22,720 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be6ed4ed40b022406f2/lava_callback.json.gz 2026-05-02 01:00:22,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:22,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be6ed4ed40b022406f2 HTTP/1.1" 200 None 2026-05-02 01:00:22,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:22,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:22,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:22,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be6ed4ed40b022406f2 HTTP/1.1" 200 None 2026-05-02 01:00:22,791 - lava_callback - INFO - Completed processing callback for node 69f54be6ed4ed40b022406f2 INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:00:27,118 - lava_callback - INFO - Received callback for node 69f54befed4ed40b02240734 INFO: 172.20.0.1:53552 - "POST /node/69f54befed4ed40b02240734 HTTP/1.0" 202 Accepted 2026-05-02 01:00:27,125 - lava_callback - INFO - Starting async job submit for node 69f54befed4ed40b02240734 2026-05-02 01:00:27,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:27,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54befed4ed40b02240734 HTTP/1.1" 200 None 2026-05-02 01:00:27,182 - lava_callback - DEBUG - Processing node 69f54befed4ed40b02240734: baseline-arm64 2026-05-02 01:00:27,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:27,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:27,640 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54befed4ed40b02240734/log.txt.gz 2026-05-02 01:00:27,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:28,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:28,037 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54befed4ed40b02240734/lava-logs.yaml 2026-05-02 01:00:28,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:28,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:28,192 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54befed4ed40b02240734/lava_callback.json.gz 2026-05-02 01:00:28,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:28,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54befed4ed40b02240734 HTTP/1.1" 200 None 2026-05-02 01:00:28,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:28,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:28,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:28,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54befed4ed40b02240734 HTTP/1.1" 200 None 2026-05-02 01:00:28,284 - lava_callback - INFO - Completed processing callback for node 69f54befed4ed40b02240734 2026-05-02 01:00:30,261 - lava_callback - INFO - Received callback for node 69f54bf9ed4ed40b02240751 INFO: 172.20.0.1:40716 - "POST /node/69f54bf9ed4ed40b02240751 HTTP/1.0" 202 Accepted 2026-05-02 01:00:30,268 - lava_callback - INFO - Starting async job submit for node 69f54bf9ed4ed40b02240751 2026-05-02 01:00:30,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:30,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf9ed4ed40b02240751 HTTP/1.1" 200 None 2026-05-02 01:00:30,307 - lava_callback - DEBUG - Processing node 69f54bf9ed4ed40b02240751: baseline-nfs-arm64 2026-05-02 01:00:30,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:31,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:31,025 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf9ed4ed40b02240751/log.txt.gz 2026-05-02 01:00:31,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:31,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:31,755 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf9ed4ed40b02240751/lava-logs.yaml 2026-05-02 01:00:31,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:31,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:31,895 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf9ed4ed40b02240751/lava_callback.json.gz 2026-05-02 01:00:31,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:31,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf9ed4ed40b02240751 HTTP/1.1" 200 None 2026-05-02 01:00:31,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:31,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:31,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:31,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf9ed4ed40b02240751 HTTP/1.1" 200 None 2026-05-02 01:00:31,983 - lava_callback - INFO - Completed processing callback for node 69f54bf9ed4ed40b02240751 2026-05-02 01:00:35,277 - lava_callback - INFO - Received callback for node 69f54bfded4ed40b0224075f INFO: 172.20.0.1:40726 - "POST /node/69f54bfded4ed40b0224075f HTTP/1.0" 202 Accepted 2026-05-02 01:00:35,283 - lava_callback - INFO - Starting async job submit for node 69f54bfded4ed40b0224075f 2026-05-02 01:00:35,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:35,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfded4ed40b0224075f HTTP/1.1" 200 None 2026-05-02 01:00:35,322 - lava_callback - DEBUG - Processing node 69f54bfded4ed40b0224075f: baseline-nfs-arm64 2026-05-02 01:00:35,515 - lava_callback - INFO - Received callback for node 69f54c00ed4ed40b0224076b INFO: 172.20.0.1:40732 - "POST /node/69f54c00ed4ed40b0224076b HTTP/1.0" 202 Accepted 2026-05-02 01:00:35,532 - lava_callback - INFO - Starting async job submit for node 69f54c00ed4ed40b0224076b 2026-05-02 01:00:35,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:35,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c00ed4ed40b0224076b HTTP/1.1" 200 None 2026-05-02 01:00:35,685 - lava_callback - DEBUG - Processing node 69f54c00ed4ed40b0224076b: baseline-nfs-arm64 2026-05-02 01:00:35,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:35,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:36,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfded4ed40b0224075f/log.txt.gz 2026-05-02 01:00:36,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:36,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:36,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:36,561 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54c00ed4ed40b0224076b/log.txt.gz INFO: 172.20.0.1:40742 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:00:36,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:36,684 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfded4ed40b0224075f/lava-logs.yaml 2026-05-02 01:00:36,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:36,897 - lava_callback - INFO - Received callback for node 69f54bf8ed4ed40b0224074e INFO: 172.20.0.1:40746 - "POST /node/69f54bf8ed4ed40b0224074e HTTP/1.0" 202 Accepted 2026-05-02 01:00:36,925 - lava_callback - INFO - Starting async job submit for node 69f54bf8ed4ed40b0224074e 2026-05-02 01:00:36,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:36,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:36,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf8ed4ed40b0224074e HTTP/1.1" 200 None 2026-05-02 01:00:36,985 - lava_callback - DEBUG - Processing node 69f54bf8ed4ed40b0224074e: baseline-nfs-arm64 2026-05-02 01:00:37,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:37,026 - lava_callback - INFO - Received callback for node 69f54bfeed4ed40b02240764 2026-05-02 01:00:37,053 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfded4ed40b0224075f/lava_callback.json.gz 2026-05-02 01:00:37,064 - lava_callback - INFO - Starting async job submit for node 69f54bfeed4ed40b02240764 INFO: 172.20.0.1:40760 - "POST /node/69f54bfeed4ed40b02240764 HTTP/1.0" 202 Accepted 2026-05-02 01:00:37,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:37,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:37,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:37,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfded4ed40b0224075f HTTP/1.1" 200 None 2026-05-02 01:00:37,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfeed4ed40b02240764 HTTP/1.1" 200 None 2026-05-02 01:00:37,271 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54c00ed4ed40b0224076b/lava-logs.yaml 2026-05-02 01:00:37,278 - lava_callback - DEBUG - Processing node 69f54bfeed4ed40b02240764: baseline-nfs-arm64 2026-05-02 01:00:37,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:37,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:37,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:37,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:37,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:37,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bfded4ed40b0224075f HTTP/1.1" 200 None 2026-05-02 01:00:37,872 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54c00ed4ed40b0224076b/lava_callback.json.gz 2026-05-02 01:00:37,900 - lava_callback - INFO - Completed processing callback for node 69f54bfded4ed40b0224075f 2026-05-02 01:00:37,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:38,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c00ed4ed40b0224076b HTTP/1.1" 200 None 2026-05-02 01:00:38,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:38,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:38,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:38,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:38,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:38,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:38,416 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfeed4ed40b02240764/log.txt.gz 2026-05-02 01:00:38,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c00ed4ed40b0224076b HTTP/1.1" 200 None 2026-05-02 01:00:38,473 - lava_callback - INFO - Completed processing callback for node 69f54c00ed4ed40b0224076b 2026-05-02 01:00:38,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:38,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf8ed4ed40b0224074e/log.txt.gz 2026-05-02 01:00:38,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:39,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:39,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:39,200 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfeed4ed40b02240764/lava-logs.yaml 2026-05-02 01:00:39,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:39,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:39,360 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfeed4ed40b02240764/lava_callback.json.gz 2026-05-02 01:00:39,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:39,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:39,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfeed4ed40b02240764 HTTP/1.1" 200 None 2026-05-02 01:00:39,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf8ed4ed40b0224074e/lava-logs.yaml 2026-05-02 01:00:39,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:39,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:39,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:39,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:39,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bfeed4ed40b02240764 HTTP/1.1" 200 None 2026-05-02 01:00:39,476 - lava_callback - INFO - Completed processing callback for node 69f54bfeed4ed40b02240764 2026-05-02 01:00:39,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:39,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf8ed4ed40b0224074e/lava_callback.json.gz 2026-05-02 01:00:39,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:39,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf8ed4ed40b0224074e HTTP/1.1" 200 None 2026-05-02 01:00:39,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:39,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:39,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:39,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf8ed4ed40b0224074e HTTP/1.1" 200 None 2026-05-02 01:00:39,724 - lava_callback - INFO - Completed processing callback for node 69f54bf8ed4ed40b0224074e 2026-05-02 01:00:42,621 - lava_callback - INFO - Received callback for node 69f54bf6ed4ed40b02240748 INFO: 172.20.0.1:52368 - "POST /node/69f54bf6ed4ed40b02240748 HTTP/1.0" 202 Accepted 2026-05-02 01:00:42,630 - lava_callback - INFO - Starting async job submit for node 69f54bf6ed4ed40b02240748 2026-05-02 01:00:42,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:42,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf6ed4ed40b02240748 HTTP/1.1" 200 None 2026-05-02 01:00:42,676 - lava_callback - DEBUG - Processing node 69f54bf6ed4ed40b02240748: baseline-nfs-arm64 2026-05-02 01:00:42,989 - lava_callback - INFO - Received callback for node 69f54be1ed4ed40b022406e6 INFO: 172.20.0.1:52370 - "POST /node/69f54be1ed4ed40b022406e6 HTTP/1.0" 202 Accepted 2026-05-02 01:00:43,014 - lava_callback - INFO - Starting async job submit for node 69f54be1ed4ed40b022406e6 2026-05-02 01:00:43,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:43,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:43,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be1ed4ed40b022406e6 HTTP/1.1" 200 None 2026-05-02 01:00:43,100 - lava_callback - DEBUG - Processing node 69f54be1ed4ed40b022406e6: baseline-arm64 2026-05-02 01:00:43,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:43,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf6ed4ed40b02240748/log.txt.gz 2026-05-02 01:00:43,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:43,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:43,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:43,671 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be1ed4ed40b022406e6/log.txt.gz 2026-05-02 01:00:43,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:43,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf6ed4ed40b02240748/lava-logs.yaml 2026-05-02 01:00:43,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:43,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:44,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:44,044 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf6ed4ed40b02240748/lava_callback.json.gz 2026-05-02 01:00:44,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:44,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:44,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be1ed4ed40b022406e6/lava-logs.yaml 2026-05-02 01:00:44,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf6ed4ed40b02240748 HTTP/1.1" 200 None 2026-05-02 01:00:44,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:44,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:44,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:44,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:44,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf6ed4ed40b02240748 HTTP/1.1" 200 None 2026-05-02 01:00:44,146 - lava_callback - INFO - Completed processing callback for node 69f54bf6ed4ed40b02240748 2026-05-02 01:00:44,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:44,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be1ed4ed40b022406e6/lava_callback.json.gz 2026-05-02 01:00:44,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:44,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be1ed4ed40b022406e6 HTTP/1.1" 200 None 2026-05-02 01:00:44,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:44,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:44,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:44,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be1ed4ed40b022406e6 HTTP/1.1" 200 None 2026-05-02 01:00:44,475 - lava_callback - INFO - Completed processing callback for node 69f54be1ed4ed40b022406e6 2026-05-02 01:00:46,408 - lava_callback - INFO - Received callback for node 69f54bfbed4ed40b02240757 INFO: 172.20.0.1:52382 - "POST /node/69f54bfbed4ed40b02240757 HTTP/1.0" 202 Accepted 2026-05-02 01:00:46,417 - lava_callback - INFO - Starting async job submit for node 69f54bfbed4ed40b02240757 2026-05-02 01:00:46,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:46,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfbed4ed40b02240757 HTTP/1.1" 200 None 2026-05-02 01:00:46,494 - lava_callback - DEBUG - Processing node 69f54bfbed4ed40b02240757: baseline-nfs-arm64 INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:00:46,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:46,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:46,945 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfbed4ed40b02240757/log.txt.gz 2026-05-02 01:00:47,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:47,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:47,350 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfbed4ed40b02240757/lava-logs.yaml 2026-05-02 01:00:47,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:00:47,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:00:47,472 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfbed4ed40b02240757/lava_callback.json.gz 2026-05-02 01:00:47,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:47,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfbed4ed40b02240757 HTTP/1.1" 200 None 2026-05-02 01:00:47,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:47,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:00:47,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:47,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bfbed4ed40b02240757 HTTP/1.1" 200 None 2026-05-02 01:00:47,564 - lava_callback - INFO - Completed processing callback for node 69f54bfbed4ed40b02240757 2026-05-02 01:00:52,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:52,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:48338 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:00:59,493 - lava_callback - INFO - Received callback for node 69f54bf3ed4ed40b02240742 INFO: 172.20.0.1:48352 - "POST /node/69f54bf3ed4ed40b02240742 HTTP/1.0" 202 Accepted 2026-05-02 01:00:59,501 - lava_callback - INFO - Starting async job submit for node 69f54bf3ed4ed40b02240742 2026-05-02 01:00:59,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:00:59,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf3ed4ed40b02240742 HTTP/1.1" 200 None 2026-05-02 01:00:59,557 - lava_callback - DEBUG - Processing node 69f54bf3ed4ed40b02240742: baseline-nfs-arm64 2026-05-02 01:00:59,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:01:00,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:01:00,045 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf3ed4ed40b02240742/log.txt.gz 2026-05-02 01:01:00,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:01:00,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:01:00,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf3ed4ed40b02240742/lava-logs.yaml 2026-05-02 01:01:00,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:01:00,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:01:00,496 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf3ed4ed40b02240742/lava_callback.json.gz 2026-05-02 01:01:00,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:00,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf3ed4ed40b02240742 HTTP/1.1" 200 None 2026-05-02 01:01:00,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:00,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:01:00,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:00,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf3ed4ed40b02240742 HTTP/1.1" 200 None 2026-05-02 01:01:00,570 - lava_callback - INFO - Completed processing callback for node 69f54bf3ed4ed40b02240742 INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46900 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:01:22,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:22,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 01:01:25,638 - lava_callback - INFO - Received callback for node 69f54846ed4ed40b0224034c INFO: 172.20.0.1:46908 - "POST /node/69f54846ed4ed40b0224034c HTTP/1.0" 202 Accepted 2026-05-02 01:01:25,645 - lava_callback - INFO - Starting async job submit for node 69f54846ed4ed40b0224034c 2026-05-02 01:01:25,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:25,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54846ed4ed40b0224034c HTTP/1.1" 200 None 2026-05-02 01:01:25,736 - lava_callback - DEBUG - Processing node 69f54846ed4ed40b0224034c: ltp-smoke 2026-05-02 01:01:26,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:01:26,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:01:26,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f54846ed4ed40b0224034c/log.txt.gz INFO: 172.20.0.1:46916 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:01:26,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:01:26,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:01:26,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f54846ed4ed40b0224034c/lava-logs.yaml 2026-05-02 01:01:26,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:01:26,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:01:26,886 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f54846ed4ed40b0224034c/lava_callback.json.gz 2026-05-02 01:01:27,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:27,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54846ed4ed40b0224034c HTTP/1.1" 200 None 2026-05-02 01:01:27,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:27,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54464ed4ed40b0223db05 HTTP/1.1" 200 None 2026-05-02 01:01:27,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:27,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54846ed4ed40b0224034c HTTP/1.1" 200 None 2026-05-02 01:01:27,211 - lava_callback - INFO - Completed processing callback for node 69f54846ed4ed40b0224034c INFO: 172.20.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51572 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:01:52,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:01:52,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41392 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:02:13,746 - lava_callback - INFO - Received callback for node 69f2aa0de2fa3d7693ba710c INFO: 172.20.0.1:49022 - "POST /node/69f2aa0de2fa3d7693ba710c HTTP/1.0" 202 Accepted 2026-05-02 01:02:13,751 - lava_callback - INFO - Starting async job submit for node 69f2aa0de2fa3d7693ba710c 2026-05-02 01:02:13,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:13,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2aa0de2fa3d7693ba710c HTTP/1.1" 200 None 2026-05-02 01:02:13,767 - lava_callback - DEBUG - Processing node 69f2aa0de2fa3d7693ba710c: baseline-arm-clabbe 2026-05-02 01:02:13,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2aa0de2fa3d7693ba710c 2026-05-02 01:02:13,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:13,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2aa0de2fa3d7693ba710c HTTP/1.1" 200 None 2026-05-02 01:02:13,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:13,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 01:02:13,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:13,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2aa0de2fa3d7693ba710c HTTP/1.1" 200 None 2026-05-02 01:02:13,822 - lava_callback - ERROR - Error processing callback for node 69f2aa0de2fa3d7693ba710c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38840 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:02:36,075 - lava_callback - INFO - Received callback for node 69f54be5ed4ed40b022406ef INFO: 172.20.0.1:52846 - "POST /node/69f54be5ed4ed40b022406ef HTTP/1.0" 202 Accepted 2026-05-02 01:02:36,081 - lava_callback - INFO - Starting async job submit for node 69f54be5ed4ed40b022406ef 2026-05-02 01:02:36,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:36,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be5ed4ed40b022406ef HTTP/1.1" 200 None 2026-05-02 01:02:36,112 - lava_callback - DEBUG - Processing node 69f54be5ed4ed40b022406ef: baseline-arm64 2026-05-02 01:02:36,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:02:36,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:02:36,567 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be5ed4ed40b022406ef/log.txt.gz INFO: 172.20.0.1:52854 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:02:36,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:02:36,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:02:36,971 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be5ed4ed40b022406ef/lava-logs.yaml 2026-05-02 01:02:36,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:02:37,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:02:37,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54be5ed4ed40b022406ef/lava_callback.json.gz 2026-05-02 01:02:37,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:37,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54be5ed4ed40b022406ef HTTP/1.1" 200 None 2026-05-02 01:02:37,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:37,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:02:37,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:37,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54be5ed4ed40b022406ef HTTP/1.1" 200 None 2026-05-02 01:02:37,205 - lava_callback - INFO - Completed processing callback for node 69f54be5ed4ed40b022406ef INFO: 172.20.0.1:51512 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:02:52,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:02:52,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33438 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:03:30,192 - lava_callback - INFO - Received callback for node 69f54bf4ed4ed40b02240745 INFO: 172.20.0.1:47726 - "POST /node/69f54bf4ed4ed40b02240745 HTTP/1.0" 202 Accepted 2026-05-02 01:03:30,199 - lava_callback - INFO - Starting async job submit for node 69f54bf4ed4ed40b02240745 2026-05-02 01:03:30,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:03:30,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf4ed4ed40b02240745 HTTP/1.1" 200 None 2026-05-02 01:03:30,231 - lava_callback - DEBUG - Processing node 69f54bf4ed4ed40b02240745: baseline-nfs-arm64 2026-05-02 01:03:30,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:03:30,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:03:30,778 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf4ed4ed40b02240745/log.txt.gz 2026-05-02 01:03:31,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:03:31,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:03:31,149 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf4ed4ed40b02240745/lava-logs.yaml 2026-05-02 01:03:31,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:03:31,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:03:31,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf4ed4ed40b02240745/lava_callback.json.gz 2026-05-02 01:03:31,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:03:31,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf4ed4ed40b02240745 HTTP/1.1" 200 None 2026-05-02 01:03:31,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:03:31,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:03:31,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:03:31,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf4ed4ed40b02240745 HTTP/1.1" 200 None 2026-05-02 01:03:31,349 - lava_callback - INFO - Completed processing callback for node 69f54bf4ed4ed40b02240745 INFO: 172.20.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52500 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:03:52,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:03:52,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:04:13,805 - lava_callback - INFO - Received callback for node 69f2aa7ae2fa3d7693ba715e INFO: 172.20.0.1:34238 - "POST /node/69f2aa7ae2fa3d7693ba715e HTTP/1.0" 202 Accepted 2026-05-02 01:04:13,810 - lava_callback - INFO - Starting async job submit for node 69f2aa7ae2fa3d7693ba715e 2026-05-02 01:04:13,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:13,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2aa7ae2fa3d7693ba715e HTTP/1.1" 200 None 2026-05-02 01:04:13,825 - lava_callback - DEBUG - Processing node 69f2aa7ae2fa3d7693ba715e: baseline-arm-clabbe 2026-05-02 01:04:13,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2aa7ae2fa3d7693ba715e 2026-05-02 01:04:13,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:13,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2aa7ae2fa3d7693ba715e HTTP/1.1" 200 None 2026-05-02 01:04:13,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:13,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 01:04:13,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:13,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2aa7ae2fa3d7693ba715e HTTP/1.1" 200 None 2026-05-02 01:04:13,874 - lava_callback - ERROR - Error processing callback for node 69f2aa7ae2fa3d7693ba715e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59224 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:04:34,066 - lava_callback - INFO - Received callback for node 69f2aa9fe2fa3d7693ba716f INFO: 172.20.0.1:47330 - "POST /node/69f2aa9fe2fa3d7693ba716f HTTP/1.0" 202 Accepted 2026-05-02 01:04:34,072 - lava_callback - INFO - Starting async job submit for node 69f2aa9fe2fa3d7693ba716f 2026-05-02 01:04:34,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:34,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2aa9fe2fa3d7693ba716f HTTP/1.1" 200 None 2026-05-02 01:04:34,087 - lava_callback - DEBUG - Processing node 69f2aa9fe2fa3d7693ba716f: baseline-arm-clabbe 2026-05-02 01:04:34,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2aa9fe2fa3d7693ba716f 2026-05-02 01:04:34,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:34,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2aa9fe2fa3d7693ba716f HTTP/1.1" 200 None 2026-05-02 01:04:34,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:34,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 01:04:34,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:04:34,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2aa9fe2fa3d7693ba716f HTTP/1.1" 200 None 2026-05-02 01:04:34,133 - lava_callback - ERROR - Error processing callback for node 69f2aa9fe2fa3d7693ba716f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39154 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:05:46,949 - lava_callback - INFO - Received callback for node 69f54c17ed4ed40b022407dd INFO: 172.20.0.1:39164 - "POST /node/69f54c17ed4ed40b022407dd HTTP/1.0" 202 Accepted 2026-05-02 01:05:46,955 - lava_callback - INFO - Starting async job submit for node 69f54c17ed4ed40b022407dd 2026-05-02 01:05:46,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:05:46,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c17ed4ed40b022407dd HTTP/1.1" 200 None 2026-05-02 01:05:46,996 - lava_callback - DEBUG - Processing node 69f54c17ed4ed40b022407dd: baseline-arm64-broonie 2026-05-02 01:05:47,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:05:47,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:05:47,337 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c17ed4ed40b022407dd/log.txt.gz 2026-05-02 01:05:47,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:05:47,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:05:47,654 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c17ed4ed40b022407dd/lava-logs.yaml 2026-05-02 01:05:47,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:05:47,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:05:47,779 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c17ed4ed40b022407dd/lava_callback.json.gz 2026-05-02 01:05:47,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:05:47,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c17ed4ed40b022407dd HTTP/1.1" 200 None 2026-05-02 01:05:47,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:05:47,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:05:47,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:05:47,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c17ed4ed40b022407dd HTTP/1.1" 200 None 2026-05-02 01:05:47,852 - lava_callback - INFO - Completed processing callback for node 69f54c17ed4ed40b022407dd 2026-05-02 01:05:52,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:05:52,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55302 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:06:31,881 - lava_callback - INFO - Received callback for node 69f549deed4ed40b02240431 INFO: 172.20.0.1:51502 - "POST /node/69f549deed4ed40b02240431 HTTP/1.0" 202 Accepted 2026-05-02 01:06:31,888 - lava_callback - INFO - Starting async job submit for node 69f549deed4ed40b02240431 2026-05-02 01:06:31,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:31,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549deed4ed40b02240431 HTTP/1.1" 200 None 2026-05-02 01:06:31,954 - lava_callback - DEBUG - Processing node 69f549deed4ed40b02240431: baseline-arm 2026-05-02 01:06:32,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:06:32,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:06:32,658 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f549deed4ed40b02240431/log.txt.gz 2026-05-02 01:06:32,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:06:33,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:06:33,175 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f549deed4ed40b02240431/lava-logs.yaml 2026-05-02 01:06:33,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:06:33,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:06:33,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f549deed4ed40b02240431/lava_callback.json.gz DEBUG: Setup failed for 69f549deed4ed40b02240431. Transitting job node result: incomplete -> fail 2026-05-02 01:06:33,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:33,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f549deed4ed40b02240431 HTTP/1.1" 200 None 2026-05-02 01:06:33,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:33,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:06:33,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:33,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f549deed4ed40b02240431 HTTP/1.1" 200 None 2026-05-02 01:06:33,664 - lava_callback - INFO - Completed processing callback for node 69f549deed4ed40b02240431 INFO: 172.20.0.1:51510 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:06:40,516 - lava_callback - INFO - Received callback for node 69f54bf2ed4ed40b0224073f INFO: 172.20.0.1:36330 - "POST /node/69f54bf2ed4ed40b0224073f HTTP/1.0" 202 Accepted 2026-05-02 01:06:40,524 - lava_callback - INFO - Starting async job submit for node 69f54bf2ed4ed40b0224073f 2026-05-02 01:06:40,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:40,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf2ed4ed40b0224073f HTTP/1.1" 200 None 2026-05-02 01:06:40,581 - lava_callback - DEBUG - Processing node 69f54bf2ed4ed40b0224073f: baseline-arm64 2026-05-02 01:06:40,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:06:41,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:06:41,036 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf2ed4ed40b0224073f/log.txt.gz 2026-05-02 01:06:41,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:06:41,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:06:41,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf2ed4ed40b0224073f/lava-logs.yaml 2026-05-02 01:06:41,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:06:41,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:06:41,535 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54bf2ed4ed40b0224073f/lava_callback.json.gz DEBUG: Setup failed for 69f54bf2ed4ed40b0224073f. Transitting job node result: incomplete -> fail 2026-05-02 01:06:41,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:41,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf2ed4ed40b0224073f HTTP/1.1" 200 None 2026-05-02 01:06:41,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:41,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:06:41,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:41,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf2ed4ed40b0224073f HTTP/1.1" 200 None 2026-05-02 01:06:41,768 - lava_callback - INFO - Completed processing callback for node 69f54bf2ed4ed40b0224073f INFO: 172.20.0.1:36340 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:06:52,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:06:52,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:07:05,474 - lava_callback - INFO - Received callback for node 69f54e12ed4ed40b022409ef INFO: 172.20.0.1:36448 - "POST /node/69f54e12ed4ed40b022409ef HTTP/1.0" 202 Accepted 2026-05-02 01:07:05,480 - lava_callback - INFO - Starting async job submit for node 69f54e12ed4ed40b022409ef 2026-05-02 01:07:05,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:05,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54e12ed4ed40b022409ef HTTP/1.1" 200 None 2026-05-02 01:07:05,521 - lava_callback - DEBUG - Processing node 69f54e12ed4ed40b022409ef: baseline-x86 2026-05-02 01:07:05,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:07:05,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:07:05,782 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f54e12ed4ed40b022409ef/log.txt.gz 2026-05-02 01:07:05,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:07:06,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:07:06,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f54e12ed4ed40b022409ef/lava-logs.yaml 2026-05-02 01:07:06,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:07:06,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:07:06,118 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f54e12ed4ed40b022409ef/lava_callback.json.gz 2026-05-02 01:07:06,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:06,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54e12ed4ed40b022409ef HTTP/1.1" 200 None 2026-05-02 01:07:06,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:06,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54482ed4ed40b0223db29 HTTP/1.1" 200 None 2026-05-02 01:07:06,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:06,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54e12ed4ed40b022409ef HTTP/1.1" 200 None 2026-05-02 01:07:06,184 - lava_callback - INFO - Completed processing callback for node 69f54e12ed4ed40b022409ef INFO: 172.20.0.1:36456 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:07:06,670 - lava_callback - INFO - Received callback for node 69f54e11ed4ed40b022409ec INFO: 172.20.0.1:36460 - "POST /node/69f54e11ed4ed40b022409ec HTTP/1.0" 202 Accepted 2026-05-02 01:07:06,676 - lava_callback - INFO - Starting async job submit for node 69f54e11ed4ed40b022409ec 2026-05-02 01:07:06,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:06,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54e11ed4ed40b022409ec HTTP/1.1" 200 None 2026-05-02 01:07:06,716 - lava_callback - DEBUG - Processing node 69f54e11ed4ed40b022409ec: baseline-x86 2026-05-02 01:07:06,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:07:07,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:07:07,044 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f54e11ed4ed40b022409ec/log.txt.gz 2026-05-02 01:07:07,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:07:07,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:07:07,296 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f54e11ed4ed40b022409ec/lava-logs.yaml 2026-05-02 01:07:07,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:07:07,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:07:07,396 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f54e11ed4ed40b022409ec/lava_callback.json.gz 2026-05-02 01:07:07,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:07,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54e11ed4ed40b022409ec HTTP/1.1" 200 None 2026-05-02 01:07:07,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:07,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54482ed4ed40b0223db29 HTTP/1.1" 200 None 2026-05-02 01:07:07,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:07,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54e11ed4ed40b022409ec HTTP/1.1" 200 None 2026-05-02 01:07:07,470 - lava_callback - INFO - Completed processing callback for node 69f54e11ed4ed40b022409ec INFO: 172.20.0.1:37394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:07:22,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:07:22,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46704 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:08:11,995 - lava_callback - INFO - Received callback for node 69f2ab72e2fa3d7693ba71f6 INFO: 172.20.0.1:41330 - "POST /node/69f2ab72e2fa3d7693ba71f6 HTTP/1.0" 202 Accepted 2026-05-02 01:08:12,003 - lava_callback - INFO - Starting async job submit for node 69f2ab72e2fa3d7693ba71f6 2026-05-02 01:08:12,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab72e2fa3d7693ba71f6 HTTP/1.1" 200 None 2026-05-02 01:08:12,020 - lava_callback - DEBUG - Processing node 69f2ab72e2fa3d7693ba71f6: baseline-arm-baylibre 2026-05-02 01:08:12,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab72e2fa3d7693ba71f6 2026-05-02 01:08:12,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab72e2fa3d7693ba71f6 HTTP/1.1" 200 None 2026-05-02 01:08:12,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:12,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab72e2fa3d7693ba71f6 HTTP/1.1" 200 None 2026-05-02 01:08:12,081 - lava_callback - ERROR - Error processing callback for node 69f2ab72e2fa3d7693ba71f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:12,685 - lava_callback - INFO - Received callback for node 69f2ab74e2fa3d7693ba71f9 INFO: 172.20.0.1:41340 - "POST /node/69f2ab74e2fa3d7693ba71f9 HTTP/1.0" 202 Accepted 2026-05-02 01:08:12,693 - lava_callback - INFO - Starting async job submit for node 69f2ab74e2fa3d7693ba71f9 2026-05-02 01:08:12,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab74e2fa3d7693ba71f9 HTTP/1.1" 200 None 2026-05-02 01:08:12,715 - lava_callback - DEBUG - Processing node 69f2ab74e2fa3d7693ba71f9: baseline-arm-baylibre 2026-05-02 01:08:12,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab74e2fa3d7693ba71f9 2026-05-02 01:08:12,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab74e2fa3d7693ba71f9 HTTP/1.1" 200 None 2026-05-02 01:08:12,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:12,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:12,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab74e2fa3d7693ba71f9 HTTP/1.1" 200 None 2026-05-02 01:08:12,787 - lava_callback - ERROR - Error processing callback for node 69f2ab74e2fa3d7693ba71f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:13,332 - lava_callback - INFO - Received callback for node 69f2ab75e2fa3d7693ba71fc INFO: 172.20.0.1:41354 - "POST /node/69f2ab75e2fa3d7693ba71fc HTTP/1.0" 202 Accepted 2026-05-02 01:08:13,339 - lava_callback - INFO - Starting async job submit for node 69f2ab75e2fa3d7693ba71fc 2026-05-02 01:08:13,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:13,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab75e2fa3d7693ba71fc HTTP/1.1" 200 None 2026-05-02 01:08:13,366 - lava_callback - DEBUG - Processing node 69f2ab75e2fa3d7693ba71fc: baseline-arm-baylibre 2026-05-02 01:08:13,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab75e2fa3d7693ba71fc 2026-05-02 01:08:13,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:13,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab75e2fa3d7693ba71fc HTTP/1.1" 200 None 2026-05-02 01:08:13,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:13,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:13,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:13,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab75e2fa3d7693ba71fc HTTP/1.1" 200 None 2026-05-02 01:08:13,432 - lava_callback - ERROR - Error processing callback for node 69f2ab75e2fa3d7693ba71fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:14,026 - lava_callback - INFO - Received callback for node 69f2ab76e2fa3d7693ba7201 INFO: 172.20.0.1:41368 - "POST /node/69f2ab76e2fa3d7693ba7201 HTTP/1.0" 202 Accepted 2026-05-02 01:08:14,036 - lava_callback - INFO - Starting async job submit for node 69f2ab76e2fa3d7693ba7201 2026-05-02 01:08:14,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab76e2fa3d7693ba7201 HTTP/1.1" 200 None 2026-05-02 01:08:14,066 - lava_callback - DEBUG - Processing node 69f2ab76e2fa3d7693ba7201: baseline-arm-baylibre 2026-05-02 01:08:14,067 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab76e2fa3d7693ba7201 2026-05-02 01:08:14,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab76e2fa3d7693ba7201 HTTP/1.1" 200 None 2026-05-02 01:08:14,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:14,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab76e2fa3d7693ba7201 HTTP/1.1" 200 None 2026-05-02 01:08:14,117 - lava_callback - ERROR - Error processing callback for node 69f2ab76e2fa3d7693ba7201: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:14,695 - lava_callback - INFO - Received callback for node 69f2ab78e2fa3d7693ba720c INFO: 172.20.0.1:41372 - "POST /node/69f2ab78e2fa3d7693ba720c HTTP/1.0" 202 Accepted 2026-05-02 01:08:14,704 - lava_callback - INFO - Starting async job submit for node 69f2ab78e2fa3d7693ba720c 2026-05-02 01:08:14,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab78e2fa3d7693ba720c HTTP/1.1" 200 None 2026-05-02 01:08:14,733 - lava_callback - DEBUG - Processing node 69f2ab78e2fa3d7693ba720c: baseline-arm-baylibre 2026-05-02 01:08:14,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab78e2fa3d7693ba720c 2026-05-02 01:08:14,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab78e2fa3d7693ba720c HTTP/1.1" 200 None 2026-05-02 01:08:14,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:14,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:14,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab78e2fa3d7693ba720c HTTP/1.1" 200 None 2026-05-02 01:08:14,808 - lava_callback - ERROR - Error processing callback for node 69f2ab78e2fa3d7693ba720c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41380 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:08:15,386 - lava_callback - INFO - Received callback for node 69f2ab7ae2fa3d7693ba720f INFO: 172.20.0.1:41394 - "POST /node/69f2ab7ae2fa3d7693ba720f HTTP/1.0" 202 Accepted 2026-05-02 01:08:15,393 - lava_callback - INFO - Starting async job submit for node 69f2ab7ae2fa3d7693ba720f 2026-05-02 01:08:15,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:15,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab7ae2fa3d7693ba720f HTTP/1.1" 200 None 2026-05-02 01:08:15,409 - lava_callback - DEBUG - Processing node 69f2ab7ae2fa3d7693ba720f: baseline-arm-baylibre 2026-05-02 01:08:15,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab7ae2fa3d7693ba720f 2026-05-02 01:08:15,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:15,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab7ae2fa3d7693ba720f HTTP/1.1" 200 None 2026-05-02 01:08:15,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:15,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:15,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:15,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab7ae2fa3d7693ba720f HTTP/1.1" 200 None 2026-05-02 01:08:15,489 - lava_callback - ERROR - Error processing callback for node 69f2ab7ae2fa3d7693ba720f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58928 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:08:26,840 - lava_callback - INFO - Received callback for node 69f54c0ced4ed40b022407bd INFO: 172.20.0.1:58938 - "POST /node/69f54c0ced4ed40b022407bd HTTP/1.0" 202 Accepted 2026-05-02 01:08:26,849 - lava_callback - INFO - Starting async job submit for node 69f54c0ced4ed40b022407bd 2026-05-02 01:08:26,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:26,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0ced4ed40b022407bd HTTP/1.1" 200 None 2026-05-02 01:08:26,905 - lava_callback - DEBUG - Processing node 69f54c0ced4ed40b022407bd: baseline-arm64-broonie 2026-05-02 01:08:27,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:08:27,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:08:27,298 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0ced4ed40b022407bd/log.txt.gz 2026-05-02 01:08:27,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:08:27,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:08:27,608 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0ced4ed40b022407bd/lava-logs.yaml 2026-05-02 01:08:27,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:08:27,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:08:27,731 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0ced4ed40b022407bd/lava_callback.json.gz 2026-05-02 01:08:27,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:27,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0ced4ed40b022407bd HTTP/1.1" 200 None 2026-05-02 01:08:27,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:27,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:08:27,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:27,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c0ced4ed40b022407bd HTTP/1.1" 200 None 2026-05-02 01:08:27,813 - lava_callback - INFO - Completed processing callback for node 69f54c0ced4ed40b022407bd 2026-05-02 01:08:34,040 - lava_callback - INFO - Received callback for node 69f2ab80e2fa3d7693ba7226 INFO: 172.20.0.1:59640 - "POST /node/69f2ab80e2fa3d7693ba7226 HTTP/1.0" 202 Accepted 2026-05-02 01:08:34,046 - lava_callback - INFO - Starting async job submit for node 69f2ab80e2fa3d7693ba7226 2026-05-02 01:08:34,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab80e2fa3d7693ba7226 HTTP/1.1" 200 None 2026-05-02 01:08:34,062 - lava_callback - DEBUG - Processing node 69f2ab80e2fa3d7693ba7226: baseline-arm-clabbe 2026-05-02 01:08:34,062 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab80e2fa3d7693ba7226 2026-05-02 01:08:34,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab80e2fa3d7693ba7226 HTTP/1.1" 200 None 2026-05-02 01:08:34,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab80e2fa3d7693ba7226 HTTP/1.1" 200 None 2026-05-02 01:08:34,113 - lava_callback - ERROR - Error processing callback for node 69f2ab80e2fa3d7693ba7226: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:34,708 - lava_callback - INFO - Received callback for node 69f2ab82e2fa3d7693ba7259 INFO: 172.20.0.1:59646 - "POST /node/69f2ab82e2fa3d7693ba7259 HTTP/1.0" 202 Accepted 2026-05-02 01:08:34,726 - lava_callback - INFO - Starting async job submit for node 69f2ab82e2fa3d7693ba7259 2026-05-02 01:08:34,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab82e2fa3d7693ba7259 HTTP/1.1" 200 None 2026-05-02 01:08:34,762 - lava_callback - DEBUG - Processing node 69f2ab82e2fa3d7693ba7259: baseline-arm-clabbe 2026-05-02 01:08:34,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab82e2fa3d7693ba7259 2026-05-02 01:08:34,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab82e2fa3d7693ba7259 HTTP/1.1" 200 None 2026-05-02 01:08:34,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:34,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:34,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab82e2fa3d7693ba7259 HTTP/1.1" 200 None 2026-05-02 01:08:34,854 - lava_callback - ERROR - Error processing callback for node 69f2ab82e2fa3d7693ba7259: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:35,411 - lava_callback - INFO - Received callback for node 69f2ab83e2fa3d7693ba725d INFO: 172.20.0.1:59658 - "POST /node/69f2ab83e2fa3d7693ba725d HTTP/1.0" 202 Accepted 2026-05-02 01:08:35,417 - lava_callback - INFO - Starting async job submit for node 69f2ab83e2fa3d7693ba725d 2026-05-02 01:08:35,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:35,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab83e2fa3d7693ba725d HTTP/1.1" 200 None 2026-05-02 01:08:35,433 - lava_callback - DEBUG - Processing node 69f2ab83e2fa3d7693ba725d: baseline-arm-clabbe 2026-05-02 01:08:35,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab83e2fa3d7693ba725d 2026-05-02 01:08:35,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:35,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab83e2fa3d7693ba725d HTTP/1.1" 200 None 2026-05-02 01:08:35,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:35,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:35,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:35,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab83e2fa3d7693ba725d HTTP/1.1" 200 None 2026-05-02 01:08:35,493 - lava_callback - ERROR - Error processing callback for node 69f2ab83e2fa3d7693ba725d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:36,093 - lava_callback - INFO - Received callback for node 69f2ab84e2fa3d7693ba7260 INFO: 172.20.0.1:59666 - "POST /node/69f2ab84e2fa3d7693ba7260 HTTP/1.0" 202 Accepted 2026-05-02 01:08:36,100 - lava_callback - INFO - Starting async job submit for node 69f2ab84e2fa3d7693ba7260 2026-05-02 01:08:36,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab84e2fa3d7693ba7260 HTTP/1.1" 200 None 2026-05-02 01:08:36,126 - lava_callback - DEBUG - Processing node 69f2ab84e2fa3d7693ba7260: baseline-arm-clabbe 2026-05-02 01:08:36,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab84e2fa3d7693ba7260 2026-05-02 01:08:36,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab84e2fa3d7693ba7260 HTTP/1.1" 200 None 2026-05-02 01:08:36,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:36,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab84e2fa3d7693ba7260 HTTP/1.1" 200 None 2026-05-02 01:08:36,188 - lava_callback - ERROR - Error processing callback for node 69f2ab84e2fa3d7693ba7260: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59680 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:08:36,762 - lava_callback - INFO - Received callback for node 69f2ab85e2fa3d7693ba7263 INFO: 172.20.0.1:59686 - "POST /node/69f2ab85e2fa3d7693ba7263 HTTP/1.0" 202 Accepted 2026-05-02 01:08:36,782 - lava_callback - INFO - Starting async job submit for node 69f2ab85e2fa3d7693ba7263 2026-05-02 01:08:36,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab85e2fa3d7693ba7263 HTTP/1.1" 200 None 2026-05-02 01:08:36,809 - lava_callback - DEBUG - Processing node 69f2ab85e2fa3d7693ba7263: baseline-arm-clabbe 2026-05-02 01:08:36,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab85e2fa3d7693ba7263 2026-05-02 01:08:36,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab85e2fa3d7693ba7263 HTTP/1.1" 200 None 2026-05-02 01:08:36,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:36,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:36,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab85e2fa3d7693ba7263 HTTP/1.1" 200 None 2026-05-02 01:08:36,908 - lava_callback - ERROR - Error processing callback for node 69f2ab85e2fa3d7693ba7263: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:37,461 - lava_callback - INFO - Received callback for node 69f2ab85e2fa3d7693ba7266 INFO: 172.20.0.1:59702 - "POST /node/69f2ab85e2fa3d7693ba7266 HTTP/1.0" 202 Accepted 2026-05-02 01:08:37,470 - lava_callback - INFO - Starting async job submit for node 69f2ab85e2fa3d7693ba7266 2026-05-02 01:08:37,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:37,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab85e2fa3d7693ba7266 HTTP/1.1" 200 None 2026-05-02 01:08:37,493 - lava_callback - DEBUG - Processing node 69f2ab85e2fa3d7693ba7266: baseline-arm-clabbe 2026-05-02 01:08:37,493 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab85e2fa3d7693ba7266 2026-05-02 01:08:37,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:37,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab85e2fa3d7693ba7266 HTTP/1.1" 200 None 2026-05-02 01:08:37,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:37,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:37,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:37,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab85e2fa3d7693ba7266 HTTP/1.1" 200 None 2026-05-02 01:08:37,556 - lava_callback - ERROR - Error processing callback for node 69f2ab85e2fa3d7693ba7266: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:38,149 - lava_callback - INFO - Received callback for node 69f2ab86e2fa3d7693ba7269 INFO: 172.20.0.1:59712 - "POST /node/69f2ab86e2fa3d7693ba7269 HTTP/1.0" 202 Accepted 2026-05-02 01:08:38,156 - lava_callback - INFO - Starting async job submit for node 69f2ab86e2fa3d7693ba7269 2026-05-02 01:08:38,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab86e2fa3d7693ba7269 HTTP/1.1" 200 None 2026-05-02 01:08:38,172 - lava_callback - DEBUG - Processing node 69f2ab86e2fa3d7693ba7269: baseline-arm-clabbe 2026-05-02 01:08:38,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab86e2fa3d7693ba7269 2026-05-02 01:08:38,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab86e2fa3d7693ba7269 HTTP/1.1" 200 None 2026-05-02 01:08:38,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:38,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab86e2fa3d7693ba7269 HTTP/1.1" 200 None 2026-05-02 01:08:38,230 - lava_callback - ERROR - Error processing callback for node 69f2ab86e2fa3d7693ba7269: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:38,871 - lava_callback - INFO - Received callback for node 69f2ab87e2fa3d7693ba7270 INFO: 172.20.0.1:59724 - "POST /node/69f2ab87e2fa3d7693ba7270 HTTP/1.0" 202 Accepted 2026-05-02 01:08:38,878 - lava_callback - INFO - Starting async job submit for node 69f2ab87e2fa3d7693ba7270 2026-05-02 01:08:38,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab87e2fa3d7693ba7270 HTTP/1.1" 200 None 2026-05-02 01:08:38,899 - lava_callback - DEBUG - Processing node 69f2ab87e2fa3d7693ba7270: baseline-arm-clabbe 2026-05-02 01:08:38,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab87e2fa3d7693ba7270 2026-05-02 01:08:38,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab87e2fa3d7693ba7270 HTTP/1.1" 200 None 2026-05-02 01:08:38,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:38,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:38,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab87e2fa3d7693ba7270 HTTP/1.1" 200 None 2026-05-02 01:08:38,950 - lava_callback - ERROR - Error processing callback for node 69f2ab87e2fa3d7693ba7270: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:08:39,546 - lava_callback - INFO - Received callback for node 69f2ab88e2fa3d7693ba7273 INFO: 172.20.0.1:59740 - "POST /node/69f2ab88e2fa3d7693ba7273 HTTP/1.0" 202 Accepted 2026-05-02 01:08:39,556 - lava_callback - INFO - Starting async job submit for node 69f2ab88e2fa3d7693ba7273 2026-05-02 01:08:39,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:39,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab88e2fa3d7693ba7273 HTTP/1.1" 200 None 2026-05-02 01:08:39,586 - lava_callback - DEBUG - Processing node 69f2ab88e2fa3d7693ba7273: baseline-arm-clabbe 2026-05-02 01:08:39,586 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ab88e2fa3d7693ba7273 2026-05-02 01:08:39,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:39,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ab88e2fa3d7693ba7273 HTTP/1.1" 200 None 2026-05-02 01:08:39,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:39,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f29fc4e2fa3d7693ba43ae HTTP/1.1" 200 None 2026-05-02 01:08:39,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:39,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ab88e2fa3d7693ba7273 HTTP/1.1" 200 None 2026-05-02 01:08:39,652 - lava_callback - ERROR - Error processing callback for node 69f2ab88e2fa3d7693ba7273: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59166 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:08:52,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:08:52,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57246 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:09:44,066 - lava_callback - INFO - Received callback for node 69f54beeed4ed40b02240731 INFO: 172.20.0.1:47082 - "POST /node/69f54beeed4ed40b02240731 HTTP/1.0" 202 Accepted 2026-05-02 01:09:44,073 - lava_callback - INFO - Starting async job submit for node 69f54beeed4ed40b02240731 2026-05-02 01:09:44,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:44,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54beeed4ed40b02240731 HTTP/1.1" 200 None 2026-05-02 01:09:44,111 - lava_callback - DEBUG - Processing node 69f54beeed4ed40b02240731: baseline-arm64 2026-05-02 01:09:44,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:09:44,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:09:44,693 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beeed4ed40b02240731/log.txt.gz 2026-05-02 01:09:45,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:09:45,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:09:45,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beeed4ed40b02240731/lava-logs.yaml 2026-05-02 01:09:45,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:09:45,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:09:45,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f54beeed4ed40b02240731/lava_callback.json.gz 2026-05-02 01:09:45,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:45,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54beeed4ed40b02240731 HTTP/1.1" 200 None 2026-05-02 01:09:45,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:45,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:09:45,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:45,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54beeed4ed40b02240731 HTTP/1.1" 200 None 2026-05-02 01:09:45,515 - lava_callback - INFO - Completed processing callback for node 69f54beeed4ed40b02240731 INFO: 172.20.0.1:47090 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:09:52,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:52,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 01:09:56,245 - lava_callback - INFO - Received callback for node 69f54bfced4ed40b0224075a INFO: 172.20.0.1:47328 - "POST /node/69f54bfced4ed40b0224075a HTTP/1.0" 202 Accepted 2026-05-02 01:09:56,252 - lava_callback - INFO - Starting async job submit for node 69f54bfced4ed40b0224075a 2026-05-02 01:09:56,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:56,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfced4ed40b0224075a HTTP/1.1" 200 None 2026-05-02 01:09:56,293 - lava_callback - DEBUG - Processing node 69f54bfced4ed40b0224075a: baseline-nfs-arm64 INFO: 172.20.0.1:47336 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:09:56,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:09:56,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:09:56,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfced4ed40b0224075a/log.txt.gz 2026-05-02 01:09:57,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:09:57,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:09:57,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfced4ed40b0224075a/lava-logs.yaml 2026-05-02 01:09:57,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:09:57,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:09:57,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bfced4ed40b0224075a/lava_callback.json.gz 2026-05-02 01:09:57,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:57,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bfced4ed40b0224075a HTTP/1.1" 200 None 2026-05-02 01:09:57,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:57,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:09:57,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:09:57,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bfced4ed40b0224075a HTTP/1.1" 200 None 2026-05-02 01:09:57,616 - lava_callback - INFO - Completed processing callback for node 69f54bfced4ed40b0224075a INFO: 172.20.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37656 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:10:22,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:10:22,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37666 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:10:34,203 - lava_callback - INFO - Received callback for node 69f2ac01e2fa3d7693ba7372 INFO: 172.20.0.1:41334 - "POST /node/69f2ac01e2fa3d7693ba7372 HTTP/1.0" 202 Accepted 2026-05-02 01:10:34,209 - lava_callback - INFO - Starting async job submit for node 69f2ac01e2fa3d7693ba7372 2026-05-02 01:10:34,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:10:34,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ac01e2fa3d7693ba7372 HTTP/1.1" 200 None 2026-05-02 01:10:34,224 - lava_callback - DEBUG - Processing node 69f2ac01e2fa3d7693ba7372: baseline-arm64-clabbe 2026-05-02 01:10:34,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ac01e2fa3d7693ba7372 2026-05-02 01:10:34,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:10:34,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ac01e2fa3d7693ba7372 HTTP/1.1" 200 None 2026-05-02 01:10:34,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:10:34,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 01:10:34,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:10:34,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ac01e2fa3d7693ba7372 HTTP/1.1" 200 None 2026-05-02 01:10:34,275 - lava_callback - ERROR - Error processing callback for node 69f2ac01e2fa3d7693ba7372: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:11:31,742 - lava_callback - INFO - Received callback for node 69f54a11ed4ed40b02240500 INFO: 172.20.0.1:41266 - "POST /node/69f54a11ed4ed40b02240500 HTTP/1.0" 202 Accepted 2026-05-02 01:11:31,789 - lava_callback - INFO - Starting async job submit for node 69f54a11ed4ed40b02240500 2026-05-02 01:11:31,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:11:31,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a11ed4ed40b02240500 HTTP/1.1" 200 None 2026-05-02 01:11:31,841 - lava_callback - DEBUG - Processing node 69f54a11ed4ed40b02240500: ltp-controllers 2026-05-02 01:11:31,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:11:32,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:11:32,061 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69f54a11ed4ed40b02240500/log.txt.gz 2026-05-02 01:11:32,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:11:32,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:11:32,237 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69f54a11ed4ed40b02240500/lava-logs.yaml 2026-05-02 01:11:32,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:11:32,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:11:32,332 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69f54a11ed4ed40b02240500/lava_callback.json.gz 2026-05-02 01:11:32,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:11:32,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a11ed4ed40b02240500 HTTP/1.1" 200 None 2026-05-02 01:11:32,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:11:32,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:11:32,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:11:32,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a11ed4ed40b02240500 HTTP/1.1" 200 None 2026-05-02 01:11:32,459 - lava_callback - INFO - Completed processing callback for node 69f54a11ed4ed40b02240500 INFO: 172.20.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44142 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:11:52,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:11:52,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58830 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:13:14,275 - lava_callback - INFO - Received callback for node 69f2ac9ae2fa3d7693ba73ce INFO: 172.20.0.1:48030 - "POST /node/69f2ac9ae2fa3d7693ba73ce HTTP/1.0" 202 Accepted 2026-05-02 01:13:14,283 - lava_callback - INFO - Starting async job submit for node 69f2ac9ae2fa3d7693ba73ce 2026-05-02 01:13:14,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:13:14,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ac9ae2fa3d7693ba73ce HTTP/1.1" 200 None 2026-05-02 01:13:14,300 - lava_callback - DEBUG - Processing node 69f2ac9ae2fa3d7693ba73ce: baseline-arm64-clabbe 2026-05-02 01:13:14,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ac9ae2fa3d7693ba73ce 2026-05-02 01:13:14,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:13:14,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ac9ae2fa3d7693ba73ce HTTP/1.1" 200 None 2026-05-02 01:13:14,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:13:14,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 01:13:14,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:13:14,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ac9ae2fa3d7693ba73ce HTTP/1.1" 200 None 2026-05-02 01:13:14,359 - lava_callback - ERROR - Error processing callback for node 69f2ac9ae2fa3d7693ba73ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33292 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:14:33,201 - lava_callback - INFO - Received callback for node 69f54a13ed4ed40b0224050b INFO: 172.20.0.1:40884 - "POST /node/69f54a13ed4ed40b0224050b HTTP/1.0" 202 Accepted 2026-05-02 01:14:33,212 - lava_callback - INFO - Starting async job submit for node 69f54a13ed4ed40b0224050b 2026-05-02 01:14:33,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:14:33,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a13ed4ed40b0224050b HTTP/1.1" 200 None 2026-05-02 01:14:33,291 - lava_callback - DEBUG - Processing node 69f54a13ed4ed40b0224050b: ltp-cve 2026-05-02 01:14:33,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:14:33,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:14:33,903 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69f54a13ed4ed40b0224050b/log.txt.gz 2026-05-02 01:14:34,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:14:34,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:14:34,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69f54a13ed4ed40b0224050b/lava-logs.yaml 2026-05-02 01:14:34,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:14:34,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:14:34,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69f54a13ed4ed40b0224050b/lava_callback.json.gz 2026-05-02 01:14:34,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:14:34,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a13ed4ed40b0224050b HTTP/1.1" 200 None 2026-05-02 01:14:34,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:14:34,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:14:34,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:14:34,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a13ed4ed40b0224050b HTTP/1.1" 200 None 2026-05-02 01:14:34,563 - lava_callback - INFO - Completed processing callback for node 69f54a13ed4ed40b0224050b INFO: 172.20.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33200 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:14:52,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:14:52,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60922 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:15:30,204 - lava_callback - INFO - Received callback for node 69f55011ed4ed40b02240b43 INFO: 172.20.0.1:50076 - "POST /node/69f55011ed4ed40b02240b43 HTTP/1.0" 202 Accepted 2026-05-02 01:15:30,210 - lava_callback - INFO - Starting async job submit for node 69f55011ed4ed40b02240b43 2026-05-02 01:15:30,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:30,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55011ed4ed40b02240b43 HTTP/1.1" 200 None 2026-05-02 01:15:30,229 - lava_callback - DEBUG - Processing node 69f55011ed4ed40b02240b43: baseline-arm 2026-05-02 01:15:30,234 - lava_callback - WARNING - Failed to upload log 2026-05-02 01:15:30,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:15:30,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:15:30,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f55011ed4ed40b02240b43/lava-logs.yaml 2026-05-02 01:15:30,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:15:30,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:15:30,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f55011ed4ed40b02240b43/lava_callback.json.gz 2026-05-02 01:15:30,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:30,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55011ed4ed40b02240b43 HTTP/1.1" 200 None 2026-05-02 01:15:30,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:30,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:15:30,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:30,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55011ed4ed40b02240b43 HTTP/1.1" 200 None 2026-05-02 01:15:30,476 - lava_callback - INFO - Completed processing callback for node 69f55011ed4ed40b02240b43 2026-05-02 01:15:32,595 - lava_callback - INFO - Received callback for node 69f54a16ed4ed40b02240511 INFO: 172.20.0.1:50090 - "POST /node/69f54a16ed4ed40b02240511 HTTP/1.0" 202 Accepted 2026-05-02 01:15:32,603 - lava_callback - INFO - Starting async job submit for node 69f54a16ed4ed40b02240511 2026-05-02 01:15:32,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:32,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a16ed4ed40b02240511 HTTP/1.1" 200 None 2026-05-02 01:15:32,643 - lava_callback - DEBUG - Processing node 69f54a16ed4ed40b02240511: ltp-fs_bind 2026-05-02 01:15:33,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:15:33,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:15:33,215 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69f54a16ed4ed40b02240511/log.txt.gz 2026-05-02 01:15:33,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:15:33,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:15:33,650 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69f54a16ed4ed40b02240511/lava-logs.yaml 2026-05-02 01:15:33,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:15:33,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:15:33,763 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69f54a16ed4ed40b02240511/lava_callback.json.gz 2026-05-02 01:15:33,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:33,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a16ed4ed40b02240511 HTTP/1.1" 200 None 2026-05-02 01:15:33,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:33,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:15:33,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:33,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a16ed4ed40b02240511 HTTP/1.1" 200 None 2026-05-02 01:15:33,849 - lava_callback - INFO - Completed processing callback for node 69f54a16ed4ed40b02240511 INFO: 172.20.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57872 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:15:52,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:15:52,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:16:54,502 - lava_callback - INFO - Received callback for node 69f2ad82e2fa3d7693ba7458 INFO: 172.20.0.1:37718 - "POST /node/69f2ad82e2fa3d7693ba7458 HTTP/1.0" 202 Accepted 2026-05-02 01:16:54,509 - lava_callback - INFO - Starting async job submit for node 69f2ad82e2fa3d7693ba7458 2026-05-02 01:16:54,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:16:54,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ad82e2fa3d7693ba7458 HTTP/1.1" 200 None 2026-05-02 01:16:54,526 - lava_callback - DEBUG - Processing node 69f2ad82e2fa3d7693ba7458: baseline-arm64-clabbe 2026-05-02 01:16:54,527 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ad82e2fa3d7693ba7458 2026-05-02 01:16:54,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:16:54,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ad82e2fa3d7693ba7458 HTTP/1.1" 200 None 2026-05-02 01:16:54,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:16:54,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 01:16:54,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:16:54,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ad82e2fa3d7693ba7458 HTTP/1.1" 200 None 2026-05-02 01:16:54,592 - lava_callback - ERROR - Error processing callback for node 69f2ad82e2fa3d7693ba7458: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46990 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:18:32,537 - lava_callback - INFO - Received callback for node 69f54a17ed4ed40b02240514 INFO: 172.20.0.1:51786 - "POST /node/69f54a17ed4ed40b02240514 HTTP/1.0" 202 Accepted 2026-05-02 01:18:32,545 - lava_callback - INFO - Starting async job submit for node 69f54a17ed4ed40b02240514 2026-05-02 01:18:32,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:18:32,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a17ed4ed40b02240514 HTTP/1.1" 200 None 2026-05-02 01:18:32,588 - lava_callback - DEBUG - Processing node 69f54a17ed4ed40b02240514: ltp-fsx 2026-05-02 01:18:32,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:18:33,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:18:33,041 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69f54a17ed4ed40b02240514/log.txt.gz 2026-05-02 01:18:33,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:18:33,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:18:33,493 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69f54a17ed4ed40b02240514/lava-logs.yaml 2026-05-02 01:18:33,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:18:33,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:18:33,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69f54a17ed4ed40b02240514/lava_callback.json.gz 2026-05-02 01:18:33,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:18:33,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a17ed4ed40b02240514 HTTP/1.1" 200 None 2026-05-02 01:18:33,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:18:33,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:18:33,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:18:33,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a17ed4ed40b02240514 HTTP/1.1" 200 None 2026-05-02 01:18:33,704 - lava_callback - INFO - Completed processing callback for node 69f54a17ed4ed40b02240514 INFO: 172.20.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39728 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:18:52,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:18:52,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35130 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:19:14,117 - lava_callback - INFO - Received callback for node 69f54bf7ed4ed40b0224074b INFO: 172.20.0.1:33286 - "POST /node/69f54bf7ed4ed40b0224074b HTTP/1.0" 202 Accepted 2026-05-02 01:19:14,126 - lava_callback - INFO - Starting async job submit for node 69f54bf7ed4ed40b0224074b 2026-05-02 01:19:14,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:14,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf7ed4ed40b0224074b HTTP/1.1" 200 None 2026-05-02 01:19:14,191 - lava_callback - DEBUG - Processing node 69f54bf7ed4ed40b0224074b: baseline-nfs-arm64 INFO: 172.20.0.1:33298 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:19:15,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:19:15,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:19:15,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf7ed4ed40b0224074b/log.txt.gz 2026-05-02 01:19:16,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:19:16,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:19:16,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf7ed4ed40b0224074b/lava-logs.yaml INFO: 172.20.0.1:33310 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:19:16,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:19:16,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:19:16,686 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f54bf7ed4ed40b0224074b/lava_callback.json.gz DEBUG: Setup failed for 69f54bf7ed4ed40b0224074b. Transitting job node result: incomplete -> fail 2026-05-02 01:19:16,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:16,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54bf7ed4ed40b0224074b HTTP/1.1" 200 None 2026-05-02 01:19:16,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:16,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:19:16,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:16,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54bf7ed4ed40b0224074b HTTP/1.1" 200 None 2026-05-02 01:19:16,983 - lava_callback - INFO - Completed processing callback for node 69f54bf7ed4ed40b0224074b INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:19:22,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:22,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36642 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:19:32,131 - lava_callback - INFO - Received callback for node 69f54a19ed4ed40b02240517 INFO: 172.20.0.1:45464 - "POST /node/69f54a19ed4ed40b02240517 HTTP/1.0" 202 Accepted 2026-05-02 01:19:32,138 - lava_callback - INFO - Starting async job submit for node 69f54a19ed4ed40b02240517 2026-05-02 01:19:32,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:32,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a19ed4ed40b02240517 HTTP/1.1" 200 None 2026-05-02 01:19:32,190 - lava_callback - DEBUG - Processing node 69f54a19ed4ed40b02240517: ltp-input 2026-05-02 01:19:32,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:19:32,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:19:32,818 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69f54a19ed4ed40b02240517/log.txt.gz 2026-05-02 01:19:33,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:19:33,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:19:33,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69f54a19ed4ed40b02240517/lava-logs.yaml 2026-05-02 01:19:33,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:19:33,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:19:33,305 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69f54a19ed4ed40b02240517/lava_callback.json.gz 2026-05-02 01:19:33,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:33,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a19ed4ed40b02240517 HTTP/1.1" 200 None 2026-05-02 01:19:33,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:33,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:19:33,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:33,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a19ed4ed40b02240517 HTTP/1.1" 200 None 2026-05-02 01:19:33,392 - lava_callback - INFO - Completed processing callback for node 69f54a19ed4ed40b02240517 2026-05-02 01:19:34,562 - lava_callback - INFO - Received callback for node 69f2ae21e2fa3d7693ba74c7 INFO: 172.20.0.1:45478 - "POST /node/69f2ae21e2fa3d7693ba74c7 HTTP/1.0" 202 Accepted 2026-05-02 01:19:34,568 - lava_callback - INFO - Starting async job submit for node 69f2ae21e2fa3d7693ba74c7 2026-05-02 01:19:34,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:34,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ae21e2fa3d7693ba74c7 HTTP/1.1" 200 None 2026-05-02 01:19:34,604 - lava_callback - DEBUG - Processing node 69f2ae21e2fa3d7693ba74c7: baseline-arm64-clabbe 2026-05-02 01:19:34,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ae21e2fa3d7693ba74c7 2026-05-02 01:19:34,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:34,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ae21e2fa3d7693ba74c7 HTTP/1.1" 200 None 2026-05-02 01:19:34,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:34,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 01:19:34,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:34,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ae21e2fa3d7693ba74c7 HTTP/1.1" 200 None 2026-05-02 01:19:34,659 - lava_callback - ERROR - Error processing callback for node 69f2ae21e2fa3d7693ba74c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58634 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:19:52,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:19:52,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39040 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:20:54,555 - lava_callback - INFO - Received callback for node 69f2ae6de2fa3d7693ba74e7 INFO: 172.20.0.1:51394 - "POST /node/69f2ae6de2fa3d7693ba74e7 HTTP/1.0" 202 Accepted 2026-05-02 01:20:54,561 - lava_callback - INFO - Starting async job submit for node 69f2ae6de2fa3d7693ba74e7 2026-05-02 01:20:54,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:20:54,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ae6de2fa3d7693ba74e7 HTTP/1.1" 200 None 2026-05-02 01:20:54,576 - lava_callback - DEBUG - Processing node 69f2ae6de2fa3d7693ba74e7: baseline-arm64-clabbe 2026-05-02 01:20:54,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ae6de2fa3d7693ba74e7 2026-05-02 01:20:54,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:20:54,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ae6de2fa3d7693ba74e7 HTTP/1.1" 200 None 2026-05-02 01:20:54,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:20:54,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 01:20:54,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:20:54,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ae6de2fa3d7693ba74e7 HTTP/1.1" 200 None 2026-05-02 01:20:54,633 - lava_callback - ERROR - Error processing callback for node 69f2ae6de2fa3d7693ba74e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33314 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:23:54,747 - lava_callback - INFO - Received callback for node 69f2af21e2fa3d7693ba755a INFO: 172.20.0.1:54046 - "POST /node/69f2af21e2fa3d7693ba755a HTTP/1.0" 202 Accepted 2026-05-02 01:23:54,753 - lava_callback - INFO - Starting async job submit for node 69f2af21e2fa3d7693ba755a 2026-05-02 01:23:54,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:23:54,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2af21e2fa3d7693ba755a HTTP/1.1" 200 None 2026-05-02 01:23:54,772 - lava_callback - DEBUG - Processing node 69f2af21e2fa3d7693ba755a: baseline-arm64-clabbe 2026-05-02 01:23:54,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2af21e2fa3d7693ba755a 2026-05-02 01:23:54,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:23:54,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2af21e2fa3d7693ba755a HTTP/1.1" 200 None 2026-05-02 01:23:54,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:23:54,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 01:23:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:23:54,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2af21e2fa3d7693ba755a HTTP/1.1" 200 None 2026-05-02 01:23:54,829 - lava_callback - ERROR - Error processing callback for node 69f2af21e2fa3d7693ba755a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37068 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:30:15,020 - lava_callback - INFO - Received callback for node 69f2b098e2fa3d7693ba7624 INFO: 172.20.0.1:36078 - "POST /node/69f2b098e2fa3d7693ba7624 HTTP/1.0" 202 Accepted 2026-05-02 01:30:15,026 - lava_callback - INFO - Starting async job submit for node 69f2b098e2fa3d7693ba7624 2026-05-02 01:30:15,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:30:15,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b098e2fa3d7693ba7624 HTTP/1.1" 200 None 2026-05-02 01:30:15,041 - lava_callback - DEBUG - Processing node 69f2b098e2fa3d7693ba7624: baseline-arm64-clabbe 2026-05-02 01:30:15,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b098e2fa3d7693ba7624 2026-05-02 01:30:15,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:30:15,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b098e2fa3d7693ba7624 HTTP/1.1" 200 None 2026-05-02 01:30:15,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:30:15,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 01:30:15,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:30:15,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b098e2fa3d7693ba7624 HTTP/1.1" 200 None 2026-05-02 01:30:15,094 - lava_callback - ERROR - Error processing callback for node 69f2b098e2fa3d7693ba7624: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48888 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:33:15,132 - lava_callback - INFO - Received callback for node 69f2b14de2fa3d7693ba7674 INFO: 172.20.0.1:53878 - "POST /node/69f2b14de2fa3d7693ba7674 HTTP/1.0" 202 Accepted 2026-05-02 01:33:15,137 - lava_callback - INFO - Starting async job submit for node 69f2b14de2fa3d7693ba7674 2026-05-02 01:33:15,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:33:15,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b14de2fa3d7693ba7674 HTTP/1.1" 200 None 2026-05-02 01:33:15,153 - lava_callback - DEBUG - Processing node 69f2b14de2fa3d7693ba7674: baseline-arm64-clabbe 2026-05-02 01:33:15,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b14de2fa3d7693ba7674 INFO: 172.20.0.1:53884 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:33:15,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:33:15,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b14de2fa3d7693ba7674 HTTP/1.1" 200 None 2026-05-02 01:33:15,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:33:15,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 01:33:15,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:33:15,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b14de2fa3d7693ba7674 HTTP/1.1" 200 None 2026-05-02 01:33:15,212 - lava_callback - ERROR - Error processing callback for node 69f2b14de2fa3d7693ba7674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40742 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:35:27,053 - lava_callback - INFO - Received callback for node 69f54c11ed4ed40b022407cd INFO: 172.20.0.1:40748 - "POST /node/69f54c11ed4ed40b022407cd HTTP/1.0" 202 Accepted 2026-05-02 01:35:27,059 - lava_callback - INFO - Starting async job submit for node 69f54c11ed4ed40b022407cd 2026-05-02 01:35:27,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:35:27,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c11ed4ed40b022407cd HTTP/1.1" 200 None 2026-05-02 01:35:27,094 - lava_callback - DEBUG - Processing node 69f54c11ed4ed40b022407cd: baseline-arm64-broonie 2026-05-02 01:35:27,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:35:27,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:35:27,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c11ed4ed40b022407cd/log.txt.gz 2026-05-02 01:35:27,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:35:27,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:35:27,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c11ed4ed40b022407cd/lava-logs.yaml 2026-05-02 01:35:27,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:35:27,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:35:27,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c11ed4ed40b022407cd/lava_callback.json.gz 2026-05-02 01:35:27,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:35:27,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c11ed4ed40b022407cd HTTP/1.1" 200 None 2026-05-02 01:35:27,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:35:27,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:35:27,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:35:28,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c11ed4ed40b022407cd HTTP/1.1" 200 None 2026-05-02 01:35:28,009 - lava_callback - INFO - Completed processing callback for node 69f54c11ed4ed40b022407cd INFO: 172.20.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37408 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:35:52,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:35:52,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55350 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:36:15,256 - lava_callback - INFO - Received callback for node 69f2b209e2fa3d7693ba76c7 INFO: 172.20.0.1:55352 - "POST /node/69f2b209e2fa3d7693ba76c7 HTTP/1.0" 202 Accepted 2026-05-02 01:36:15,261 - lava_callback - INFO - Starting async job submit for node 69f2b209e2fa3d7693ba76c7 2026-05-02 01:36:15,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:36:15,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b209e2fa3d7693ba76c7 HTTP/1.1" 200 None 2026-05-02 01:36:15,276 - lava_callback - DEBUG - Processing node 69f2b209e2fa3d7693ba76c7: baseline-arm64-clabbe 2026-05-02 01:36:15,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b209e2fa3d7693ba76c7 2026-05-02 01:36:15,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:36:15,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b209e2fa3d7693ba76c7 HTTP/1.1" 200 None 2026-05-02 01:36:15,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:36:15,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 01:36:15,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:36:15,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b209e2fa3d7693ba76c7 HTTP/1.1" 200 None 2026-05-02 01:36:15,321 - lava_callback - ERROR - Error processing callback for node 69f2b209e2fa3d7693ba76c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37762 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:38:55,351 - lava_callback - INFO - Received callback for node 69f2b29ee2fa3d7693ba7710 INFO: 172.20.0.1:52840 - "POST /node/69f2b29ee2fa3d7693ba7710 HTTP/1.0" 202 Accepted 2026-05-02 01:38:55,356 - lava_callback - INFO - Starting async job submit for node 69f2b29ee2fa3d7693ba7710 2026-05-02 01:38:55,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:38:55,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b29ee2fa3d7693ba7710 HTTP/1.1" 200 None 2026-05-02 01:38:55,372 - lava_callback - DEBUG - Processing node 69f2b29ee2fa3d7693ba7710: baseline-arm64-clabbe 2026-05-02 01:38:55,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b29ee2fa3d7693ba7710 2026-05-02 01:38:55,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:38:55,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b29ee2fa3d7693ba7710 HTTP/1.1" 200 None 2026-05-02 01:38:55,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:38:55,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 01:38:55,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:38:55,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b29ee2fa3d7693ba7710 HTTP/1.1" 200 None 2026-05-02 01:38:55,425 - lava_callback - ERROR - Error processing callback for node 69f2b29ee2fa3d7693ba7710: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:39:55,346 - lava_callback - INFO - Received callback for node 69f2b2e8e2fa3d7693ba773f INFO: 172.20.0.1:51420 - "POST /node/69f2b2e8e2fa3d7693ba773f HTTP/1.0" 202 Accepted 2026-05-02 01:39:55,352 - lava_callback - INFO - Starting async job submit for node 69f2b2e8e2fa3d7693ba773f 2026-05-02 01:39:55,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:39:55,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b2e8e2fa3d7693ba773f HTTP/1.1" 200 None 2026-05-02 01:39:55,372 - lava_callback - DEBUG - Processing node 69f2b2e8e2fa3d7693ba773f: baseline-arm64-clabbe 2026-05-02 01:39:55,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b2e8e2fa3d7693ba773f 2026-05-02 01:39:55,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:39:55,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b2e8e2fa3d7693ba773f HTTP/1.1" 200 None 2026-05-02 01:39:55,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:39:55,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 01:39:55,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:39:55,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b2e8e2fa3d7693ba773f HTTP/1.1" 200 None 2026-05-02 01:39:55,439 - lava_callback - ERROR - Error processing callback for node 69f2b2e8e2fa3d7693ba773f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36134 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:41:15,421 - lava_callback - INFO - Received callback for node 69f2b334e2fa3d7693ba775c INFO: 172.20.0.1:36144 - "POST /node/69f2b334e2fa3d7693ba775c HTTP/1.0" 202 Accepted 2026-05-02 01:41:15,427 - lava_callback - INFO - Starting async job submit for node 69f2b334e2fa3d7693ba775c 2026-05-02 01:41:15,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:15,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b334e2fa3d7693ba775c HTTP/1.1" 200 None 2026-05-02 01:41:15,442 - lava_callback - DEBUG - Processing node 69f2b334e2fa3d7693ba775c: baseline-arm64-clabbe 2026-05-02 01:41:15,442 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b334e2fa3d7693ba775c 2026-05-02 01:41:15,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:15,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b334e2fa3d7693ba775c HTTP/1.1" 200 None 2026-05-02 01:41:15,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:15,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 01:41:15,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:15,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b334e2fa3d7693ba775c HTTP/1.1" 200 None 2026-05-02 01:41:15,488 - lava_callback - ERROR - Error processing callback for node 69f2b334e2fa3d7693ba775c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53902 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:41:28,395 - lava_callback - INFO - Received callback for node 69f55631ed4ed40b02240e22 INFO: 172.20.0.1:53916 - "POST /node/69f55631ed4ed40b02240e22 HTTP/1.0" 202 Accepted 2026-05-02 01:41:28,401 - lava_callback - INFO - Starting async job submit for node 69f55631ed4ed40b02240e22 2026-05-02 01:41:28,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:28,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55631ed4ed40b02240e22 HTTP/1.1" 200 None 2026-05-02 01:41:28,417 - lava_callback - DEBUG - Processing node 69f55631ed4ed40b02240e22: baseline-nfs-arm64 2026-05-02 01:41:28,418 - lava_callback - WARNING - Failed to upload log 2026-05-02 01:41:28,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:41:28,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:41:28,500 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f55631ed4ed40b02240e22/lava-logs.yaml 2026-05-02 01:41:28,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:41:28,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:41:28,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f55631ed4ed40b02240e22/lava_callback.json.gz 2026-05-02 01:41:28,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:28,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55631ed4ed40b02240e22 HTTP/1.1" 200 None 2026-05-02 01:41:28,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:28,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:41:28,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:28,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55631ed4ed40b02240e22 HTTP/1.1" 200 None 2026-05-02 01:41:28,658 - lava_callback - INFO - Completed processing callback for node 69f55631ed4ed40b02240e22 INFO: 172.20.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:41:52,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:41:53,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52002 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:42:35,508 - lava_callback - INFO - Received callback for node 69f2b380e2fa3d7693ba7788 INFO: 172.20.0.1:38818 - "POST /node/69f2b380e2fa3d7693ba7788 HTTP/1.0" 202 Accepted 2026-05-02 01:42:35,516 - lava_callback - INFO - Starting async job submit for node 69f2b380e2fa3d7693ba7788 2026-05-02 01:42:35,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:42:35,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b380e2fa3d7693ba7788 HTTP/1.1" 200 None 2026-05-02 01:42:35,538 - lava_callback - DEBUG - Processing node 69f2b380e2fa3d7693ba7788: baseline-arm64-clabbe 2026-05-02 01:42:35,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b380e2fa3d7693ba7788 2026-05-02 01:42:35,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:42:35,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b380e2fa3d7693ba7788 HTTP/1.1" 200 None 2026-05-02 01:42:35,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:42:35,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 01:42:35,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:42:35,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b380e2fa3d7693ba7788 HTTP/1.1" 200 None 2026-05-02 01:42:35,583 - lava_callback - ERROR - Error processing callback for node 69f2b380e2fa3d7693ba7788: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37432 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:44:33,188 - lava_callback - INFO - Received callback for node 69f54c14ed4ed40b022407d5 INFO: 172.20.0.1:39068 - "POST /node/69f54c14ed4ed40b022407d5 HTTP/1.0" 202 Accepted 2026-05-02 01:44:33,194 - lava_callback - INFO - Starting async job submit for node 69f54c14ed4ed40b022407d5 2026-05-02 01:44:33,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:33,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c14ed4ed40b022407d5 HTTP/1.1" 200 None 2026-05-02 01:44:33,282 - lava_callback - DEBUG - Processing node 69f54c14ed4ed40b022407d5: baseline-arm64-broonie 2026-05-02 01:44:33,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:44:33,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:44:33,765 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c14ed4ed40b022407d5/log.txt.gz 2026-05-02 01:44:33,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:44:34,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:44:34,081 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c14ed4ed40b022407d5/lava-logs.yaml 2026-05-02 01:44:34,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:44:34,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:44:34,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c14ed4ed40b022407d5/lava_callback.json.gz 2026-05-02 01:44:34,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:34,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c14ed4ed40b022407d5 HTTP/1.1" 200 None 2026-05-02 01:44:34,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:34,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:44:34,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:34,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c14ed4ed40b022407d5 HTTP/1.1" 200 None 2026-05-02 01:44:34,314 - lava_callback - INFO - Completed processing callback for node 69f54c14ed4ed40b022407d5 INFO: 172.20.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46290 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:44:53,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:53,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 01:44:55,628 - lava_callback - INFO - Received callback for node 69f2b415e2fa3d7693ba77d6 INFO: 172.20.0.1:55370 - "POST /node/69f2b415e2fa3d7693ba77d6 HTTP/1.0" 202 Accepted 2026-05-02 01:44:55,634 - lava_callback - INFO - Starting async job submit for node 69f2b415e2fa3d7693ba77d6 2026-05-02 01:44:55,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:55,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b415e2fa3d7693ba77d6 HTTP/1.1" 200 None 2026-05-02 01:44:55,649 - lava_callback - DEBUG - Processing node 69f2b415e2fa3d7693ba77d6: baseline-arm64-clabbe 2026-05-02 01:44:55,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b415e2fa3d7693ba77d6 2026-05-02 01:44:55,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:55,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b415e2fa3d7693ba77d6 HTTP/1.1" 200 None 2026-05-02 01:44:55,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:55,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 01:44:55,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:44:55,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b415e2fa3d7693ba77d6 HTTP/1.1" 200 None 2026-05-02 01:44:55,693 - lava_callback - ERROR - Error processing callback for node 69f2b415e2fa3d7693ba77d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36834 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:47:35,668 - lava_callback - INFO - Received callback for node 69f2b4b0e2fa3d7693ba7823 INFO: 172.20.0.1:35122 - "POST /node/69f2b4b0e2fa3d7693ba7823 HTTP/1.0" 202 Accepted 2026-05-02 01:47:35,674 - lava_callback - INFO - Starting async job submit for node 69f2b4b0e2fa3d7693ba7823 2026-05-02 01:47:35,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:35,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b4b0e2fa3d7693ba7823 HTTP/1.1" 200 None 2026-05-02 01:47:35,689 - lava_callback - DEBUG - Processing node 69f2b4b0e2fa3d7693ba7823: baseline-arm64-clabbe 2026-05-02 01:47:35,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b4b0e2fa3d7693ba7823 2026-05-02 01:47:35,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:35,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b4b0e2fa3d7693ba7823 HTTP/1.1" 200 None 2026-05-02 01:47:35,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:35,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 01:47:35,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:35,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b4b0e2fa3d7693ba7823 HTTP/1.1" 200 None 2026-05-02 01:47:35,766 - lava_callback - ERROR - Error processing callback for node 69f2b4b0e2fa3d7693ba7823: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56542 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:47:55,474 - lava_callback - INFO - Received callback for node 69f55797ed4ed40b02240ebd INFO: 172.20.0.1:52168 - "POST /node/69f55797ed4ed40b02240ebd HTTP/1.0" 202 Accepted 2026-05-02 01:47:55,481 - lava_callback - INFO - Starting async job submit for node 69f55797ed4ed40b02240ebd 2026-05-02 01:47:55,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:55,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55797ed4ed40b02240ebd HTTP/1.1" 200 None 2026-05-02 01:47:55,533 - lava_callback - DEBUG - Processing node 69f55797ed4ed40b02240ebd: baseline-arm64 2026-05-02 01:47:55,675 - lava_callback - WARNING - Failed to upload log 2026-05-02 01:47:55,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:47:55,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:47:55,922 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55797ed4ed40b02240ebd/lava-logs.yaml 2026-05-02 01:47:55,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:47:56,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:47:56,023 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55797ed4ed40b02240ebd/lava_callback.json.gz 2026-05-02 01:47:56,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:56,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55797ed4ed40b02240ebd HTTP/1.1" 200 None 2026-05-02 01:47:56,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:56,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:47:56,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:47:56,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55797ed4ed40b02240ebd HTTP/1.1" 200 None 2026-05-02 01:47:56,159 - lava_callback - INFO - Completed processing callback for node 69f55797ed4ed40b02240ebd INFO: 172.20.0.1:52178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35076 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:48:23,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:48:23,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41760 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:48:55,793 - lava_callback - INFO - Received callback for node 69f2b4fde2fa3d7693ba7848 INFO: 172.20.0.1:44130 - "POST /node/69f2b4fde2fa3d7693ba7848 HTTP/1.0" 202 Accepted 2026-05-02 01:48:55,798 - lava_callback - INFO - Starting async job submit for node 69f2b4fde2fa3d7693ba7848 2026-05-02 01:48:55,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:48:55,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b4fde2fa3d7693ba7848 HTTP/1.1" 200 None 2026-05-02 01:48:55,813 - lava_callback - DEBUG - Processing node 69f2b4fde2fa3d7693ba7848: baseline-arm64-clabbe 2026-05-02 01:48:55,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b4fde2fa3d7693ba7848 2026-05-02 01:48:55,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:48:55,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b4fde2fa3d7693ba7848 HTTP/1.1" 200 None 2026-05-02 01:48:55,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:48:55,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 01:48:55,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:48:55,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b4fde2fa3d7693ba7848 HTTP/1.1" 200 None 2026-05-02 01:48:55,865 - lava_callback - ERROR - Error processing callback for node 69f2b4fde2fa3d7693ba7848: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43310 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:50:02,896 - lava_callback - INFO - Received callback for node 69f557e8ed4ed40b02240ee5 INFO: 172.20.0.1:37038 - "POST /node/69f557e8ed4ed40b02240ee5 HTTP/1.0" 202 Accepted 2026-05-02 01:50:02,902 - lava_callback - INFO - Starting async job submit for node 69f557e8ed4ed40b02240ee5 2026-05-02 01:50:02,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:02,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f557e8ed4ed40b02240ee5 HTTP/1.1" 200 None 2026-05-02 01:50:02,955 - lava_callback - DEBUG - Processing node 69f557e8ed4ed40b02240ee5: baseline-arm64 2026-05-02 01:50:03,154 - lava_callback - WARNING - Failed to upload log 2026-05-02 01:50:03,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:50:03,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:50:03,410 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f557e8ed4ed40b02240ee5/lava-logs.yaml 2026-05-02 01:50:03,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:50:03,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:50:03,524 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f557e8ed4ed40b02240ee5/lava_callback.json.gz 2026-05-02 01:50:03,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:03,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f557e8ed4ed40b02240ee5 HTTP/1.1" 200 None 2026-05-02 01:50:03,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:03,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:50:03,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:03,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f557e8ed4ed40b02240ee5 HTTP/1.1" 200 None 2026-05-02 01:50:03,663 - lava_callback - INFO - Completed processing callback for node 69f557e8ed4ed40b02240ee5 INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42110 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:50:15,853 - lava_callback - INFO - Received callback for node 69f2b54ae2fa3d7693ba7870 INFO: 172.20.0.1:42120 - "POST /node/69f2b54ae2fa3d7693ba7870 HTTP/1.0" 202 Accepted 2026-05-02 01:50:15,859 - lava_callback - INFO - Starting async job submit for node 69f2b54ae2fa3d7693ba7870 2026-05-02 01:50:15,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:15,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b54ae2fa3d7693ba7870 HTTP/1.1" 200 None 2026-05-02 01:50:15,881 - lava_callback - DEBUG - Processing node 69f2b54ae2fa3d7693ba7870: baseline-arm64-clabbe 2026-05-02 01:50:15,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b54ae2fa3d7693ba7870 2026-05-02 01:50:15,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:15,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b54ae2fa3d7693ba7870 HTTP/1.1" 200 None 2026-05-02 01:50:15,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:15,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 01:50:15,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:15,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b54ae2fa3d7693ba7870 HTTP/1.1" 200 None 2026-05-02 01:50:15,932 - lava_callback - ERROR - Error processing callback for node 69f2b54ae2fa3d7693ba7870: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50804 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:50:23,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:50:23,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36548 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:51:14,206 - lava_callback - INFO - Received callback for node 69f2b584e2fa3d7693ba7891 INFO: 172.20.0.1:34796 - "POST /node/69f2b584e2fa3d7693ba7891 HTTP/1.0" 202 Accepted 2026-05-02 01:51:14,211 - lava_callback - INFO - Starting async job submit for node 69f2b584e2fa3d7693ba7891 2026-05-02 01:51:14,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:14,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b584e2fa3d7693ba7891 HTTP/1.1" 200 None 2026-05-02 01:51:14,226 - lava_callback - DEBUG - Processing node 69f2b584e2fa3d7693ba7891: baseline-arm-baylibre 2026-05-02 01:51:14,226 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b584e2fa3d7693ba7891 2026-05-02 01:51:14,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:14,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b584e2fa3d7693ba7891 HTTP/1.1" 200 None 2026-05-02 01:51:14,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:14,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 01:51:14,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:14,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b584e2fa3d7693ba7891 HTTP/1.1" 200 None 2026-05-02 01:51:14,284 - lava_callback - ERROR - Error processing callback for node 69f2b584e2fa3d7693ba7891: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37164 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:51:54,077 - lava_callback - INFO - Received callback for node 69f2b5a8e2fa3d7693ba78a0 INFO: 172.20.0.1:50874 - "POST /node/69f2b5a8e2fa3d7693ba78a0 HTTP/1.0" 202 Accepted 2026-05-02 01:51:54,082 - lava_callback - INFO - Starting async job submit for node 69f2b5a8e2fa3d7693ba78a0 2026-05-02 01:51:54,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:54,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b5a8e2fa3d7693ba78a0 HTTP/1.1" 200 None 2026-05-02 01:51:54,101 - lava_callback - DEBUG - Processing node 69f2b5a8e2fa3d7693ba78a0: baseline-arm-baylibre 2026-05-02 01:51:54,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b5a8e2fa3d7693ba78a0 2026-05-02 01:51:54,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:54,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b5a8e2fa3d7693ba78a0 HTTP/1.1" 200 None 2026-05-02 01:51:54,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:54,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 01:51:54,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:51:54,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b5a8e2fa3d7693ba78a0 HTTP/1.1" 200 None 2026-05-02 01:51:54,147 - lava_callback - ERROR - Error processing callback for node 69f2b5a8e2fa3d7693ba78a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33506 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:52:14,100 - lava_callback - INFO - Received callback for node 69f2b5cce2fa3d7693ba78ac INFO: 172.20.0.1:33414 - "POST /node/69f2b5cce2fa3d7693ba78ac HTTP/1.0" 202 Accepted 2026-05-02 01:52:14,106 - lava_callback - INFO - Starting async job submit for node 69f2b5cce2fa3d7693ba78ac 2026-05-02 01:52:14,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:14,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b5cce2fa3d7693ba78ac HTTP/1.1" 200 None 2026-05-02 01:52:14,120 - lava_callback - DEBUG - Processing node 69f2b5cce2fa3d7693ba78ac: baseline-arm-baylibre 2026-05-02 01:52:14,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b5cce2fa3d7693ba78ac 2026-05-02 01:52:14,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:14,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b5cce2fa3d7693ba78ac HTTP/1.1" 200 None 2026-05-02 01:52:14,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:14,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 01:52:14,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:14,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b5cce2fa3d7693ba78ac HTTP/1.1" 200 None 2026-05-02 01:52:14,170 - lava_callback - ERROR - Error processing callback for node 69f2b5cce2fa3d7693ba78ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:52:33,146 - lava_callback - INFO - Received callback for node 69f54a1aed4ed40b0224051a INFO: 172.20.0.1:34000 - "POST /node/69f54a1aed4ed40b0224051a HTTP/1.0" 202 Accepted 2026-05-02 01:52:33,154 - lava_callback - INFO - Starting async job submit for node 69f54a1aed4ed40b0224051a 2026-05-02 01:52:33,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:33,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1aed4ed40b0224051a HTTP/1.1" 200 None 2026-05-02 01:52:33,207 - lava_callback - DEBUG - Processing node 69f54a1aed4ed40b0224051a: ltp-ipc 2026-05-02 01:52:33,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:33,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:33,679 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69f54a1aed4ed40b0224051a/log.txt.gz 2026-05-02 01:52:33,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:34,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:34,062 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69f54a1aed4ed40b0224051a/lava-logs.yaml 2026-05-02 01:52:34,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:34,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:34,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69f54a1aed4ed40b0224051a/lava_callback.json.gz 2026-05-02 01:52:34,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:34,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1aed4ed40b0224051a HTTP/1.1" 200 None 2026-05-02 01:52:34,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:34,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:52:34,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:34,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a1aed4ed40b0224051a HTTP/1.1" 200 None 2026-05-02 01:52:34,264 - lava_callback - INFO - Completed processing callback for node 69f54a1aed4ed40b0224051a INFO: 172.20.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60022 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:52:53,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:53,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 01:52:53,408 - lava_callback - INFO - Received callback for node 69f54a1ced4ed40b0224051d INFO: 172.20.0.1:39676 - "POST /node/69f54a1ced4ed40b0224051d HTTP/1.0" 202 Accepted 2026-05-02 01:52:53,414 - lava_callback - INFO - Starting async job submit for node 69f54a1ced4ed40b0224051d 2026-05-02 01:52:53,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:53,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1ced4ed40b0224051d HTTP/1.1" 200 None 2026-05-02 01:52:53,453 - lava_callback - DEBUG - Processing node 69f54a1ced4ed40b0224051d: ltp-pty 2026-05-02 01:52:53,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:53,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:53,889 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69f54a1ced4ed40b0224051d/log.txt.gz 2026-05-02 01:52:54,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:54,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:54,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69f54a1ced4ed40b0224051d/lava-logs.yaml 2026-05-02 01:52:54,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:54,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:54,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69f54a1ced4ed40b0224051d/lava_callback.json.gz 2026-05-02 01:52:54,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:54,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1ced4ed40b0224051d HTTP/1.1" 200 None 2026-05-02 01:52:54,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:54,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:52:54,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:54,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a1ced4ed40b0224051d HTTP/1.1" 200 None 2026-05-02 01:52:54,498 - lava_callback - INFO - Completed processing callback for node 69f54a1ced4ed40b0224051d 2026-05-02 01:52:54,863 - lava_callback - INFO - Received callback for node 69f54c0fed4ed40b022407c3 INFO: 172.20.0.1:39682 - "POST /node/69f54c0fed4ed40b022407c3 HTTP/1.0" 202 Accepted 2026-05-02 01:52:54,870 - lava_callback - INFO - Starting async job submit for node 69f54c0fed4ed40b022407c3 2026-05-02 01:52:54,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:54,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0fed4ed40b022407c3 HTTP/1.1" 200 None 2026-05-02 01:52:54,908 - lava_callback - DEBUG - Processing node 69f54c0fed4ed40b022407c3: baseline-arm64-broonie 2026-05-02 01:52:55,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:55,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:55,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0fed4ed40b022407c3/log.txt.gz 2026-05-02 01:52:55,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:55,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:55,553 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0fed4ed40b022407c3/lava-logs.yaml 2026-05-02 01:52:55,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:52:55,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:52:55,646 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0fed4ed40b022407c3/lava_callback.json.gz 2026-05-02 01:52:55,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:55,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0fed4ed40b022407c3 HTTP/1.1" 200 None 2026-05-02 01:52:55,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:55,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:52:55,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:52:55,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c0fed4ed40b022407c3 HTTP/1.1" 200 None 2026-05-02 01:52:55,727 - lava_callback - INFO - Completed processing callback for node 69f54c0fed4ed40b022407c3 INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45558 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:53:07,500 - lava_callback - INFO - Received callback for node 69f54a1eed4ed40b02240523 INFO: 172.20.0.1:45572 - "POST /node/69f54a1eed4ed40b02240523 HTTP/1.0" 202 Accepted 2026-05-02 01:53:07,506 - lava_callback - INFO - Starting async job submit for node 69f54a1eed4ed40b02240523 2026-05-02 01:53:07,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:07,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1eed4ed40b02240523 HTTP/1.1" 200 None 2026-05-02 01:53:07,549 - lava_callback - DEBUG - Processing node 69f54a1eed4ed40b02240523: ltp-smoke 2026-05-02 01:53:07,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:53:08,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:53:08,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f54a1eed4ed40b02240523/log.txt.gz 2026-05-02 01:53:08,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:53:08,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:53:08,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f54a1eed4ed40b02240523/lava-logs.yaml 2026-05-02 01:53:08,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:53:08,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:53:08,568 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f54a1eed4ed40b02240523/lava_callback.json.gz 2026-05-02 01:53:08,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:08,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1eed4ed40b02240523 HTTP/1.1" 200 None 2026-05-02 01:53:08,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:08,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:53:08,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:08,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a1eed4ed40b02240523 HTTP/1.1" 200 None 2026-05-02 01:53:08,657 - lava_callback - INFO - Completed processing callback for node 69f54a1eed4ed40b02240523 INFO: 172.20.0.1:35482 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:53:15,984 - lava_callback - INFO - Received callback for node 69f2b608e2fa3d7693ba78c0 INFO: 172.20.0.1:35486 - "POST /node/69f2b608e2fa3d7693ba78c0 HTTP/1.0" 202 Accepted 2026-05-02 01:53:15,990 - lava_callback - INFO - Starting async job submit for node 69f2b608e2fa3d7693ba78c0 2026-05-02 01:53:15,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:16,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b608e2fa3d7693ba78c0 HTTP/1.1" 200 None 2026-05-02 01:53:16,005 - lava_callback - DEBUG - Processing node 69f2b608e2fa3d7693ba78c0: baseline-arm64-clabbe 2026-05-02 01:53:16,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b608e2fa3d7693ba78c0 2026-05-02 01:53:16,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:16,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b608e2fa3d7693ba78c0 HTTP/1.1" 200 None 2026-05-02 01:53:16,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:16,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 01:53:16,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:16,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b608e2fa3d7693ba78c0 HTTP/1.1" 200 None 2026-05-02 01:53:16,067 - lava_callback - ERROR - Error processing callback for node 69f2b608e2fa3d7693ba78c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36986 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:53:23,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:23,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36988 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:53:33,445 - lava_callback - INFO - Received callback for node 69f54a1ded4ed40b02240520 INFO: 172.20.0.1:45456 - "POST /node/69f54a1ded4ed40b02240520 HTTP/1.0" 202 Accepted 2026-05-02 01:53:33,452 - lava_callback - INFO - Starting async job submit for node 69f54a1ded4ed40b02240520 2026-05-02 01:53:33,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:33,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1ded4ed40b02240520 HTTP/1.1" 200 None 2026-05-02 01:53:33,490 - lava_callback - DEBUG - Processing node 69f54a1ded4ed40b02240520: ltp-sched 2026-05-02 01:53:33,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:53:33,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:53:33,954 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69f54a1ded4ed40b02240520/log.txt.gz 2026-05-02 01:53:34,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:53:34,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:53:34,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69f54a1ded4ed40b02240520/lava-logs.yaml 2026-05-02 01:53:34,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:53:34,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:53:34,456 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69f54a1ded4ed40b02240520/lava_callback.json.gz 2026-05-02 01:53:34,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:34,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a1ded4ed40b02240520 HTTP/1.1" 200 None 2026-05-02 01:53:34,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:53:34,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:34,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a1ded4ed40b02240520 HTTP/1.1" 200 None 2026-05-02 01:53:34,546 - lava_callback - INFO - Completed processing callback for node 69f54a1ded4ed40b02240520 INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37714 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:53:53,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:53:53,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34332 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:54:34,312 - lava_callback - INFO - Received callback for node 69f2b645e2fa3d7693ba78e2 INFO: 172.20.0.1:39356 - "POST /node/69f2b645e2fa3d7693ba78e2 HTTP/1.0" 202 Accepted 2026-05-02 01:54:34,317 - lava_callback - INFO - Starting async job submit for node 69f2b645e2fa3d7693ba78e2 2026-05-02 01:54:34,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:54:34,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b645e2fa3d7693ba78e2 HTTP/1.1" 200 None 2026-05-02 01:54:34,331 - lava_callback - DEBUG - Processing node 69f2b645e2fa3d7693ba78e2: baseline-arm-baylibre 2026-05-02 01:54:34,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b645e2fa3d7693ba78e2 2026-05-02 01:54:34,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:54:34,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b645e2fa3d7693ba78e2 HTTP/1.1" 200 None 2026-05-02 01:54:34,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:54:34,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 01:54:34,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:54:34,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b645e2fa3d7693ba78e2 HTTP/1.1" 200 None 2026-05-02 01:54:34,381 - lava_callback - ERROR - Error processing callback for node 69f2b645e2fa3d7693ba78e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33180 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 01:55:16,068 - lava_callback - INFO - Received callback for node 69f2b67ae2fa3d7693ba78f4 INFO: 172.20.0.1:33192 - "POST /node/69f2b67ae2fa3d7693ba78f4 HTTP/1.0" 202 Accepted 2026-05-02 01:55:16,073 - lava_callback - INFO - Starting async job submit for node 69f2b67ae2fa3d7693ba78f4 2026-05-02 01:55:16,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:55:16,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b67ae2fa3d7693ba78f4 HTTP/1.1" 200 None 2026-05-02 01:55:16,103 - lava_callback - DEBUG - Processing node 69f2b67ae2fa3d7693ba78f4: baseline-arm64-clabbe 2026-05-02 01:55:16,103 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b67ae2fa3d7693ba78f4 2026-05-02 01:55:16,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:55:16,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b67ae2fa3d7693ba78f4 HTTP/1.1" 200 None 2026-05-02 01:55:16,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:55:16,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 01:55:16,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:55:16,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b67ae2fa3d7693ba78f4 HTTP/1.1" 200 None 2026-05-02 01:55:16,157 - lava_callback - ERROR - Error processing callback for node 69f2b67ae2fa3d7693ba78f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39954 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:56:33,264 - lava_callback - INFO - Received callback for node 69f54a21ed4ed40b02240529 INFO: 172.20.0.1:39226 - "POST /node/69f54a21ed4ed40b02240529 HTTP/1.0" 202 Accepted 2026-05-02 01:56:33,271 - lava_callback - INFO - Starting async job submit for node 69f54a21ed4ed40b02240529 2026-05-02 01:56:33,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:33,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a21ed4ed40b02240529 HTTP/1.1" 200 None 2026-05-02 01:56:33,317 - lava_callback - DEBUG - Processing node 69f54a21ed4ed40b02240529: ltp-syscalls-ipc 2026-05-02 01:56:33,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:56:33,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:56:33,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69f54a21ed4ed40b02240529/log.txt.gz 2026-05-02 01:56:34,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:56:34,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:56:34,175 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69f54a21ed4ed40b02240529/lava-logs.yaml 2026-05-02 01:56:34,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:56:34,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:56:34,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69f54a21ed4ed40b02240529/lava_callback.json.gz 2026-05-02 01:56:34,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:34,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a21ed4ed40b02240529 HTTP/1.1" 200 None 2026-05-02 01:56:34,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:34,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:56:34,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:34,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a21ed4ed40b02240529 HTTP/1.1" 200 None 2026-05-02 01:56:34,383 - lava_callback - INFO - Completed processing callback for node 69f54a21ed4ed40b02240529 INFO: 172.20.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54846 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:56:53,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:53,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 01:56:53,597 - lava_callback - INFO - Received callback for node 69f54a23ed4ed40b0224052c INFO: 172.20.0.1:43764 - "POST /node/69f54a23ed4ed40b0224052c HTTP/1.0" 202 Accepted 2026-05-02 01:56:53,603 - lava_callback - INFO - Starting async job submit for node 69f54a23ed4ed40b0224052c 2026-05-02 01:56:53,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:53,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a23ed4ed40b0224052c HTTP/1.1" 200 None 2026-05-02 01:56:53,641 - lava_callback - DEBUG - Processing node 69f54a23ed4ed40b0224052c: ltp-timers 2026-05-02 01:56:54,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:56:54,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:56:54,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69f54a23ed4ed40b0224052c/log.txt.gz 2026-05-02 01:56:54,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:56:54,387 - lava_callback - INFO - Received callback for node 69f2b6d8e2fa3d7693ba7926 INFO: 172.20.0.1:43768 - "POST /node/69f2b6d8e2fa3d7693ba7926 HTTP/1.0" 202 Accepted 2026-05-02 01:56:54,393 - lava_callback - INFO - Starting async job submit for node 69f2b6d8e2fa3d7693ba7926 2026-05-02 01:56:54,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:54,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b6d8e2fa3d7693ba7926 HTTP/1.1" 200 None 2026-05-02 01:56:54,409 - lava_callback - DEBUG - Processing node 69f2b6d8e2fa3d7693ba7926: baseline-arm-baylibre 2026-05-02 01:56:54,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b6d8e2fa3d7693ba7926 2026-05-02 01:56:54,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:54,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b6d8e2fa3d7693ba7926 HTTP/1.1" 200 None 2026-05-02 01:56:54,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:54,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 01:56:54,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:54,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:56:54,516 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69f54a23ed4ed40b0224052c/lava-logs.yaml 2026-05-02 01:56:54,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b6d8e2fa3d7693ba7926 HTTP/1.1" 200 None 2026-05-02 01:56:54,519 - lava_callback - ERROR - Error processing callback for node 69f2b6d8e2fa3d7693ba7926: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 01:56:54,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:56:54,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:56:54,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69f54a23ed4ed40b0224052c/lava_callback.json.gz 2026-05-02 01:56:54,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:54,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a23ed4ed40b0224052c HTTP/1.1" 200 None 2026-05-02 01:56:54,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:54,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:56:54,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:56:54,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a23ed4ed40b0224052c HTTP/1.1" 200 None 2026-05-02 01:56:54,748 - lava_callback - INFO - Completed processing callback for node 69f54a23ed4ed40b0224052c INFO: 172.20.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51424 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:57:07,839 - lava_callback - INFO - Received callback for node 69f54a24ed4ed40b0224052f INFO: 172.20.0.1:51438 - "POST /node/69f54a24ed4ed40b0224052f HTTP/1.0" 202 Accepted 2026-05-02 01:57:07,849 - lava_callback - INFO - Starting async job submit for node 69f54a24ed4ed40b0224052f 2026-05-02 01:57:07,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:07,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a24ed4ed40b0224052f HTTP/1.1" 200 None 2026-05-02 01:57:07,897 - lava_callback - DEBUG - Processing node 69f54a24ed4ed40b0224052f: ltp-watchqueue 2026-05-02 01:57:08,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:57:08,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:57:08,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69f54a24ed4ed40b0224052f/log.txt.gz 2026-05-02 01:57:08,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:57:08,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:57:08,674 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69f54a24ed4ed40b0224052f/lava-logs.yaml 2026-05-02 01:57:08,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:57:08,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:57:08,793 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69f54a24ed4ed40b0224052f/lava_callback.json.gz 2026-05-02 01:57:08,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:08,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54a24ed4ed40b0224052f HTTP/1.1" 200 None 2026-05-02 01:57:08,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:08,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 01:57:08,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:08,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54a24ed4ed40b0224052f HTTP/1.1" 200 None 2026-05-02 01:57:08,881 - lava_callback - INFO - Completed processing callback for node 69f54a24ed4ed40b0224052f INFO: 172.20.0.1:52750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38466 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:57:23,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:23,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38470 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:57:33,147 - lava_callback - INFO - Received callback for node 69f54c16ed4ed40b022407d8 INFO: 172.20.0.1:45120 - "POST /node/69f54c16ed4ed40b022407d8 HTTP/1.0" 202 Accepted 2026-05-02 01:57:33,153 - lava_callback - INFO - Starting async job submit for node 69f54c16ed4ed40b022407d8 2026-05-02 01:57:33,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:33,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c16ed4ed40b022407d8 HTTP/1.1" 200 None 2026-05-02 01:57:33,191 - lava_callback - DEBUG - Processing node 69f54c16ed4ed40b022407d8: baseline-arm64-broonie 2026-05-02 01:57:33,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:57:33,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:57:33,510 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c16ed4ed40b022407d8/log.txt.gz 2026-05-02 01:57:33,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:57:33,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:57:33,791 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c16ed4ed40b022407d8/lava-logs.yaml 2026-05-02 01:57:33,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 01:57:33,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 01:57:33,905 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c16ed4ed40b022407d8/lava_callback.json.gz 2026-05-02 01:57:33,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:33,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c16ed4ed40b022407d8 HTTP/1.1" 200 None 2026-05-02 01:57:33,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:33,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 01:57:33,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:33,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c16ed4ed40b022407d8 HTTP/1.1" 200 None 2026-05-02 01:57:33,995 - lava_callback - INFO - Completed processing callback for node 69f54c16ed4ed40b022407d8 2026-05-02 01:57:36,170 - lava_callback - INFO - Received callback for node 69f2b70be2fa3d7693ba7939 INFO: 172.20.0.1:45124 - "POST /node/69f2b70be2fa3d7693ba7939 HTTP/1.0" 202 Accepted 2026-05-02 01:57:36,177 - lava_callback - INFO - Starting async job submit for node 69f2b70be2fa3d7693ba7939 2026-05-02 01:57:36,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:36,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b70be2fa3d7693ba7939 HTTP/1.1" 200 None 2026-05-02 01:57:36,192 - lava_callback - DEBUG - Processing node 69f2b70be2fa3d7693ba7939: baseline-arm64-clabbe 2026-05-02 01:57:36,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b70be2fa3d7693ba7939 2026-05-02 01:57:36,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:36,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b70be2fa3d7693ba7939 HTTP/1.1" 200 None 2026-05-02 01:57:36,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:36,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 01:57:36,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:36,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b70be2fa3d7693ba7939 HTTP/1.1" 200 None 2026-05-02 01:57:36,242 - lava_callback - ERROR - Error processing callback for node 69f2b70be2fa3d7693ba7939: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57526 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:57:53,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:57:53,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46514 - "GET / HTTP/1.0" 200 OK 2026-05-02 01:58:34,519 - lava_callback - INFO - Received callback for node 69f2b746e2fa3d7693ba7959 INFO: 172.20.0.1:48818 - "POST /node/69f2b746e2fa3d7693ba7959 HTTP/1.0" 202 Accepted 2026-05-02 01:58:34,524 - lava_callback - INFO - Starting async job submit for node 69f2b746e2fa3d7693ba7959 2026-05-02 01:58:34,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:58:34,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b746e2fa3d7693ba7959 HTTP/1.1" 200 None 2026-05-02 01:58:34,539 - lava_callback - DEBUG - Processing node 69f2b746e2fa3d7693ba7959: baseline-arm-baylibre 2026-05-02 01:58:34,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b746e2fa3d7693ba7959 2026-05-02 01:58:34,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:58:34,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b746e2fa3d7693ba7959 HTTP/1.1" 200 None 2026-05-02 01:58:34,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:58:34,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 01:58:34,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 01:58:34,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b746e2fa3d7693ba7959 HTTP/1.1" 200 None 2026-05-02 01:58:34,592 - lava_callback - ERROR - Error processing callback for node 69f2b746e2fa3d7693ba7959: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44336 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:03:56,713 - lava_callback - INFO - Received callback for node 69f2b882e2fa3d7693ba79e3 INFO: 172.20.0.1:44342 - "POST /node/69f2b882e2fa3d7693ba79e3 HTTP/1.0" 202 Accepted 2026-05-02 02:03:56,719 - lava_callback - INFO - Starting async job submit for node 69f2b882e2fa3d7693ba79e3 2026-05-02 02:03:56,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:03:56,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b882e2fa3d7693ba79e3 HTTP/1.1" 200 None 2026-05-02 02:03:56,734 - lava_callback - DEBUG - Processing node 69f2b882e2fa3d7693ba79e3: baseline-arm64-clabbe 2026-05-02 02:03:56,734 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b882e2fa3d7693ba79e3 2026-05-02 02:03:56,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:03:56,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b882e2fa3d7693ba79e3 HTTP/1.1" 200 None 2026-05-02 02:03:56,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:03:56,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69e9650e8137cb226ce1ca7a HTTP/1.1" 200 None 2026-05-02 02:03:56,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:03:56,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b882e2fa3d7693ba79e3 HTTP/1.1" 200 None 2026-05-02 02:03:56,784 - lava_callback - ERROR - Error processing callback for node 69f2b882e2fa3d7693ba79e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41812 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:04:56,363 - lava_callback - INFO - Received callback for node 69f2b8c0e2fa3d7693ba7a03 INFO: 172.20.0.1:51744 - "POST /node/69f2b8c0e2fa3d7693ba7a03 HTTP/1.0" 202 Accepted 2026-05-02 02:04:56,368 - lava_callback - INFO - Starting async job submit for node 69f2b8c0e2fa3d7693ba7a03 2026-05-02 02:04:56,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:04:56,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b8c0e2fa3d7693ba7a03 HTTP/1.1" 200 None 2026-05-02 02:04:56,384 - lava_callback - DEBUG - Processing node 69f2b8c0e2fa3d7693ba7a03: baseline-arm-clabbe 2026-05-02 02:04:56,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b8c0e2fa3d7693ba7a03 2026-05-02 02:04:56,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:04:56,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b8c0e2fa3d7693ba7a03 HTTP/1.1" 200 None 2026-05-02 02:04:56,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:04:56,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:04:56,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:04:56,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b8c0e2fa3d7693ba7a03 HTTP/1.1" 200 None 2026-05-02 02:04:56,430 - lava_callback - ERROR - Error processing callback for node 69f2b8c0e2fa3d7693ba7a03: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:05:36,387 - lava_callback - INFO - Received callback for node 69f2b8e4e2fa3d7693ba7a14 INFO: 172.20.0.1:39582 - "POST /node/69f2b8e4e2fa3d7693ba7a14 HTTP/1.0" 202 Accepted 2026-05-02 02:05:36,394 - lava_callback - INFO - Starting async job submit for node 69f2b8e4e2fa3d7693ba7a14 2026-05-02 02:05:36,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:05:36,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b8e4e2fa3d7693ba7a14 HTTP/1.1" 200 None 2026-05-02 02:05:36,410 - lava_callback - DEBUG - Processing node 69f2b8e4e2fa3d7693ba7a14: baseline-arm-clabbe 2026-05-02 02:05:36,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b8e4e2fa3d7693ba7a14 2026-05-02 02:05:36,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:05:36,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b8e4e2fa3d7693ba7a14 HTTP/1.1" 200 None 2026-05-02 02:05:36,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:05:36,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:05:36,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:39584 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:05:36,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b8e4e2fa3d7693ba7a14 HTTP/1.1" 200 None 2026-05-02 02:05:36,482 - lava_callback - ERROR - Error processing callback for node 69f2b8e4e2fa3d7693ba7a14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44244 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:06:36,472 - lava_callback - INFO - Received callback for node 69f2b918e2fa3d7693ba7a23 INFO: 172.20.0.1:44260 - "POST /node/69f2b918e2fa3d7693ba7a23 HTTP/1.0" 202 Accepted 2026-05-02 02:06:36,477 - lava_callback - INFO - Starting async job submit for node 69f2b918e2fa3d7693ba7a23 2026-05-02 02:06:36,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:06:36,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b918e2fa3d7693ba7a23 HTTP/1.1" 200 None 2026-05-02 02:06:36,500 - lava_callback - DEBUG - Processing node 69f2b918e2fa3d7693ba7a23: baseline-arm64-clabbe 2026-05-02 02:06:36,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b918e2fa3d7693ba7a23 2026-05-02 02:06:36,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:06:36,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b918e2fa3d7693ba7a23 HTTP/1.1" 200 None 2026-05-02 02:06:36,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:06:36,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:06:36,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:06:36,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b918e2fa3d7693ba7a23 HTTP/1.1" 200 None 2026-05-02 02:06:36,547 - lava_callback - ERROR - Error processing callback for node 69f2b918e2fa3d7693ba7a23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:08:47,565 - lava_callback - INFO - Received callback for node 69f55b09ed4ed40b02241074 INFO: 172.20.0.1:35220 - "POST /node/69f55b09ed4ed40b02241074 HTTP/1.0" 202 Accepted 2026-05-02 02:08:47,575 - lava_callback - INFO - Starting async job submit for node 69f55b09ed4ed40b02241074 2026-05-02 02:08:47,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:47,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55b09ed4ed40b02241074 HTTP/1.1" 200 None 2026-05-02 02:08:47,650 - lava_callback - DEBUG - Processing node 69f55b09ed4ed40b02241074: baseline-arm64 2026-05-02 02:08:48,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:08:48,139 - lava_callback - INFO - Received callback for node 69f5588aed4ed40b02240f34 INFO: 172.20.0.1:35228 - "POST /node/69f5588aed4ed40b02240f34 HTTP/1.0" 202 Accepted 2026-05-02 02:08:48,146 - lava_callback - INFO - Starting async job submit for node 69f5588aed4ed40b02240f34 2026-05-02 02:08:48,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:08:48,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:48,209 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55b09ed4ed40b02241074/log.txt.gz 2026-05-02 02:08:48,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5588aed4ed40b02240f34 HTTP/1.1" 200 None 2026-05-02 02:08:48,307 - lava_callback - DEBUG - Processing node 69f5588aed4ed40b02240f34: ltp-smoke 2026-05-02 02:08:48,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:08:48,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:08:48,896 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55b09ed4ed40b02241074/lava-logs.yaml 2026-05-02 02:08:48,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:08:48,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:08:49,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:08:49,049 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f5588aed4ed40b02240f34/log.txt.gz 2026-05-02 02:08:49,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:08:49,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55b09ed4ed40b02241074/lava_callback.json.gz DEBUG: Setup failed for 69f55b09ed4ed40b02241074. Transitting job node result: incomplete -> fail 2026-05-02 02:08:49,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:49,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55b09ed4ed40b02241074 HTTP/1.1" 200 None 2026-05-02 02:08:49,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:49,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 02:08:49,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:49,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:08:49,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55b09ed4ed40b02241074 HTTP/1.1" 200 None 2026-05-02 02:08:49,610 - lava_callback - INFO - Completed processing callback for node 69f55b09ed4ed40b02241074 2026-05-02 02:08:49,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:08:49,707 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f5588aed4ed40b02240f34/lava-logs.yaml 2026-05-02 02:08:49,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:08:49,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:08:49,826 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f5588aed4ed40b02240f34/lava_callback.json.gz 2026-05-02 02:08:49,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:49,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5588aed4ed40b02240f34 HTTP/1.1" 200 None 2026-05-02 02:08:49,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:49,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54464ed4ed40b0223db05 HTTP/1.1" 200 None 2026-05-02 02:08:49,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:49,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f5588aed4ed40b02240f34 HTTP/1.1" 200 None 2026-05-02 02:08:50,101 - lava_callback - INFO - Completed processing callback for node 69f5588aed4ed40b02240f34 2026-05-02 02:08:53,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:08:53,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:09:09,634 - lava_callback - INFO - Received callback for node 69f54c13ed4ed40b022407d2 INFO: 172.20.0.1:45800 - "POST /node/69f54c13ed4ed40b022407d2 HTTP/1.0" 202 Accepted 2026-05-02 02:09:09,639 - lava_callback - INFO - Starting async job submit for node 69f54c13ed4ed40b022407d2 2026-05-02 02:09:09,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:09,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c13ed4ed40b022407d2 HTTP/1.1" 200 None 2026-05-02 02:09:09,678 - lava_callback - DEBUG - Processing node 69f54c13ed4ed40b022407d2: baseline-arm64-broonie 2026-05-02 02:09:09,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:09:10,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:09:10,027 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c13ed4ed40b022407d2/log.txt.gz 2026-05-02 02:09:10,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:09:10,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:09:10,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c13ed4ed40b022407d2/lava-logs.yaml 2026-05-02 02:09:10,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:09:10,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:09:10,445 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c13ed4ed40b022407d2/lava_callback.json.gz 2026-05-02 02:09:10,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:10,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c13ed4ed40b022407d2 HTTP/1.1" 200 None 2026-05-02 02:09:10,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:10,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 02:09:10,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:10,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c13ed4ed40b022407d2 HTTP/1.1" 200 None 2026-05-02 02:09:10,517 - lava_callback - INFO - Completed processing callback for node 69f54c13ed4ed40b022407d2 INFO: 172.20.0.1:50444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50456 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:09:16,635 - lava_callback - INFO - Received callback for node 69f2b9c5e2fa3d7693ba7a7a INFO: 172.20.0.1:50468 - "POST /node/69f2b9c5e2fa3d7693ba7a7a HTTP/1.0" 202 Accepted 2026-05-02 02:09:16,640 - lava_callback - INFO - Starting async job submit for node 69f2b9c5e2fa3d7693ba7a7a 2026-05-02 02:09:16,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:16,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b9c5e2fa3d7693ba7a7a HTTP/1.1" 200 None 2026-05-02 02:09:16,666 - lava_callback - DEBUG - Processing node 69f2b9c5e2fa3d7693ba7a7a: baseline-arm-clabbe 2026-05-02 02:09:16,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b9c5e2fa3d7693ba7a7a 2026-05-02 02:09:16,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:16,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b9c5e2fa3d7693ba7a7a HTTP/1.1" 200 None 2026-05-02 02:09:16,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:16,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:09:16,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:16,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b9c5e2fa3d7693ba7a7a HTTP/1.1" 200 None 2026-05-02 02:09:16,716 - lava_callback - ERROR - Error processing callback for node 69f2b9c5e2fa3d7693ba7a7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:09:23,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:09:23,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45158 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:10:16,699 - lava_callback - INFO - Received callback for node 69f2b9f9e2fa3d7693ba7a8b INFO: 172.20.0.1:45170 - "POST /node/69f2b9f9e2fa3d7693ba7a8b HTTP/1.0" 202 Accepted 2026-05-02 02:10:16,706 - lava_callback - INFO - Starting async job submit for node 69f2b9f9e2fa3d7693ba7a8b 2026-05-02 02:10:16,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:10:16,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b9f9e2fa3d7693ba7a8b HTTP/1.1" 200 None 2026-05-02 02:10:16,720 - lava_callback - DEBUG - Processing node 69f2b9f9e2fa3d7693ba7a8b: baseline-arm64-clabbe 2026-05-02 02:10:16,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2b9f9e2fa3d7693ba7a8b 2026-05-02 02:10:16,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:10:16,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2b9f9e2fa3d7693ba7a8b HTTP/1.1" 200 None 2026-05-02 02:10:16,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:10:16,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 02:10:16,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:10:16,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2b9f9e2fa3d7693ba7a8b HTTP/1.1" 200 None 2026-05-02 02:10:16,764 - lava_callback - ERROR - Error processing callback for node 69f2b9f9e2fa3d7693ba7a8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57744 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:11:16,752 - lava_callback - INFO - Received callback for node 69f2ba39e2fa3d7693ba7aa6 INFO: 172.20.0.1:57750 - "POST /node/69f2ba39e2fa3d7693ba7aa6 HTTP/1.0" 202 Accepted 2026-05-02 02:11:16,758 - lava_callback - INFO - Starting async job submit for node 69f2ba39e2fa3d7693ba7aa6 2026-05-02 02:11:16,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:16,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ba39e2fa3d7693ba7aa6 HTTP/1.1" 200 None 2026-05-02 02:11:16,773 - lava_callback - DEBUG - Processing node 69f2ba39e2fa3d7693ba7aa6: baseline-arm-clabbe 2026-05-02 02:11:16,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ba39e2fa3d7693ba7aa6 2026-05-02 02:11:16,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:16,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ba39e2fa3d7693ba7aa6 HTTP/1.1" 200 None 2026-05-02 02:11:16,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:16,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:11:16,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:16,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ba39e2fa3d7693ba7aa6 HTTP/1.1" 200 None 2026-05-02 02:11:16,819 - lava_callback - ERROR - Error processing callback for node 69f2ba39e2fa3d7693ba7aa6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:11:56,800 - lava_callback - INFO - Received callback for node 69f2ba5fe2fa3d7693ba7ab4 INFO: 172.20.0.1:34674 - "POST /node/69f2ba5fe2fa3d7693ba7ab4 HTTP/1.0" 202 Accepted 2026-05-02 02:11:56,805 - lava_callback - INFO - Starting async job submit for node 69f2ba5fe2fa3d7693ba7ab4 2026-05-02 02:11:56,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:56,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ba5fe2fa3d7693ba7ab4 HTTP/1.1" 200 None 2026-05-02 02:11:56,820 - lava_callback - DEBUG - Processing node 69f2ba5fe2fa3d7693ba7ab4: baseline-arm-clabbe 2026-05-02 02:11:56,820 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ba5fe2fa3d7693ba7ab4 2026-05-02 02:11:56,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:56,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ba5fe2fa3d7693ba7ab4 HTTP/1.1" 200 None 2026-05-02 02:11:56,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:56,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:11:56,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:11:56,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ba5fe2fa3d7693ba7ab4 HTTP/1.1" 200 None 2026-05-02 02:11:56,866 - lava_callback - ERROR - Error processing callback for node 69f2ba5fe2fa3d7693ba7ab4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57360 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:13:16,836 - lava_callback - INFO - Received callback for node 69f2baa8e2fa3d7693ba7ad6 INFO: 172.20.0.1:57370 - "POST /node/69f2baa8e2fa3d7693ba7ad6 HTTP/1.0" 202 Accepted 2026-05-02 02:13:16,842 - lava_callback - INFO - Starting async job submit for node 69f2baa8e2fa3d7693ba7ad6 2026-05-02 02:13:16,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:16,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2baa8e2fa3d7693ba7ad6 HTTP/1.1" 200 None 2026-05-02 02:13:16,873 - lava_callback - DEBUG - Processing node 69f2baa8e2fa3d7693ba7ad6: baseline-arm-clabbe 2026-05-02 02:13:16,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2baa8e2fa3d7693ba7ad6 2026-05-02 02:13:16,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:16,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2baa8e2fa3d7693ba7ad6 HTTP/1.1" 200 None 2026-05-02 02:13:16,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:16,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:13:16,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:16,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2baa8e2fa3d7693ba7ad6 HTTP/1.1" 200 None 2026-05-02 02:13:16,919 - lava_callback - ERROR - Error processing callback for node 69f2baa8e2fa3d7693ba7ad6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:13:56,835 - lava_callback - INFO - Received callback for node 69f2badce2fa3d7693ba7ae3 INFO: 172.20.0.1:40640 - "POST /node/69f2badce2fa3d7693ba7ae3 HTTP/1.0" 202 Accepted 2026-05-02 02:13:56,843 - lava_callback - INFO - Starting async job submit for node 69f2badce2fa3d7693ba7ae3 2026-05-02 02:13:56,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:56,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2badce2fa3d7693ba7ae3 HTTP/1.1" 200 None 2026-05-02 02:13:56,879 - lava_callback - DEBUG - Processing node 69f2badce2fa3d7693ba7ae3: baseline-arm64-clabbe 2026-05-02 02:13:56,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2badce2fa3d7693ba7ae3 2026-05-02 02:13:56,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:56,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2badce2fa3d7693ba7ae3 HTTP/1.1" 200 None 2026-05-02 02:13:56,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:56,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 02:13:56,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:13:56,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2badce2fa3d7693ba7ae3 HTTP/1.1" 200 None 2026-05-02 02:13:56,930 - lava_callback - ERROR - Error processing callback for node 69f2badce2fa3d7693ba7ae3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55634 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:16:16,936 - lava_callback - INFO - Received callback for node 69f2bb68e2fa3d7693ba7b25 INFO: 172.20.0.1:55642 - "POST /node/69f2bb68e2fa3d7693ba7b25 HTTP/1.0" 202 Accepted 2026-05-02 02:16:16,941 - lava_callback - INFO - Starting async job submit for node 69f2bb68e2fa3d7693ba7b25 2026-05-02 02:16:16,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:16:16,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bb68e2fa3d7693ba7b25 HTTP/1.1" 200 None 2026-05-02 02:16:16,956 - lava_callback - DEBUG - Processing node 69f2bb68e2fa3d7693ba7b25: baseline-arm-clabbe 2026-05-02 02:16:16,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bb68e2fa3d7693ba7b25 2026-05-02 02:16:16,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:16:16,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bb68e2fa3d7693ba7b25 HTTP/1.1" 200 None 2026-05-02 02:16:16,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:16:16,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:16:16,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:16:17,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bb68e2fa3d7693ba7b25 HTTP/1.1" 200 None 2026-05-02 02:16:17,010 - lava_callback - ERROR - Error processing callback for node 69f2bb68e2fa3d7693ba7b25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58162 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:17:17,021 - lava_callback - INFO - Received callback for node 69f2bb9de2fa3d7693ba7b38 INFO: 172.20.0.1:58164 - "POST /node/69f2bb9de2fa3d7693ba7b38 HTTP/1.0" 202 Accepted 2026-05-02 02:17:17,027 - lava_callback - INFO - Starting async job submit for node 69f2bb9de2fa3d7693ba7b38 2026-05-02 02:17:17,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:17:17,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bb9de2fa3d7693ba7b38 HTTP/1.1" 200 None 2026-05-02 02:17:17,047 - lava_callback - DEBUG - Processing node 69f2bb9de2fa3d7693ba7b38: baseline-arm64-clabbe 2026-05-02 02:17:17,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bb9de2fa3d7693ba7b38 2026-05-02 02:17:17,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:17:17,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bb9de2fa3d7693ba7b38 HTTP/1.1" 200 None 2026-05-02 02:17:17,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:17:17,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 02:17:17,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:17:17,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bb9de2fa3d7693ba7b38 HTTP/1.1" 200 None 2026-05-02 02:17:17,093 - lava_callback - ERROR - Error processing callback for node 69f2bb9de2fa3d7693ba7b38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43968 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:18:02,172 - lava_callback - INFO - Received callback for node 69f55added4ed40b02241066 INFO: 172.20.0.1:45784 - "POST /node/69f55added4ed40b02241066 HTTP/1.0" 202 Accepted 2026-05-02 02:18:02,179 - lava_callback - INFO - Starting async job submit for node 69f55added4ed40b02241066 2026-05-02 02:18:02,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:02,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55added4ed40b02241066 HTTP/1.1" 200 None 2026-05-02 02:18:02,247 - lava_callback - DEBUG - Processing node 69f55added4ed40b02241066: baseline-arm 2026-05-02 02:18:02,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:18:03,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:18:03,126 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f55added4ed40b02241066/log.txt.gz 2026-05-02 02:18:03,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:18:03,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:18:03,640 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f55added4ed40b02241066/lava-logs.yaml 2026-05-02 02:18:03,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:18:03,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:18:03,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f55added4ed40b02241066/lava_callback.json.gz DEBUG: Setup failed for 69f55added4ed40b02241066. Transitting job node result: incomplete -> fail 2026-05-02 02:18:03,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:03,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55added4ed40b02241066 HTTP/1.1" 200 None 2026-05-02 02:18:03,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:03,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 02:18:03,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:03,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55added4ed40b02241066 HTTP/1.1" 200 None 2026-05-02 02:18:04,014 - lava_callback - INFO - Completed processing callback for node 69f55added4ed40b02241066 INFO: 172.20.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:18:17,068 - lava_callback - INFO - Received callback for node 69f2bbdce2fa3d7693ba7b56 INFO: 172.20.0.1:51746 - "POST /node/69f2bbdce2fa3d7693ba7b56 HTTP/1.0" 202 Accepted 2026-05-02 02:18:17,073 - lava_callback - INFO - Starting async job submit for node 69f2bbdce2fa3d7693ba7b56 2026-05-02 02:18:17,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:17,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bbdce2fa3d7693ba7b56 HTTP/1.1" 200 None 2026-05-02 02:18:17,089 - lava_callback - DEBUG - Processing node 69f2bbdce2fa3d7693ba7b56: baseline-arm-clabbe 2026-05-02 02:18:17,089 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bbdce2fa3d7693ba7b56 2026-05-02 02:18:17,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:17,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bbdce2fa3d7693ba7b56 HTTP/1.1" 200 None 2026-05-02 02:18:17,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:17,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:18:17,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:17,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bbdce2fa3d7693ba7b56 HTTP/1.1" 200 None 2026-05-02 02:18:17,152 - lava_callback - ERROR - Error processing callback for node 69f2bbdce2fa3d7693ba7b56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51218 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:18:23,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:18:23,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39380 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:19:57,078 - lava_callback - INFO - Received callback for node 69f2bc37e2fa3d7693ba7b75 INFO: 172.20.0.1:39396 - "POST /node/69f2bc37e2fa3d7693ba7b75 HTTP/1.0" 202 Accepted 2026-05-02 02:19:57,084 - lava_callback - INFO - Starting async job submit for node 69f2bc37e2fa3d7693ba7b75 2026-05-02 02:19:57,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:19:57,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bc37e2fa3d7693ba7b75 HTTP/1.1" 200 None 2026-05-02 02:19:57,105 - lava_callback - DEBUG - Processing node 69f2bc37e2fa3d7693ba7b75: baseline-arm64-clabbe 2026-05-02 02:19:57,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bc37e2fa3d7693ba7b75 2026-05-02 02:19:57,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:19:57,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bc37e2fa3d7693ba7b75 HTTP/1.1" 200 None 2026-05-02 02:19:57,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:19:57,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69e9650e8137cb226ce1ca7a HTTP/1.1" 200 None 2026-05-02 02:19:57,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:19:57,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bc37e2fa3d7693ba7b75 HTTP/1.1" 200 None 2026-05-02 02:19:57,160 - lava_callback - ERROR - Error processing callback for node 69f2bc37e2fa3d7693ba7b75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48798 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:20:57,157 - lava_callback - INFO - Received callback for node 69f2bc77e2fa3d7693ba7b97 INFO: 172.20.0.1:48810 - "POST /node/69f2bc77e2fa3d7693ba7b97 HTTP/1.0" 202 Accepted 2026-05-02 02:20:57,162 - lava_callback - INFO - Starting async job submit for node 69f2bc77e2fa3d7693ba7b97 2026-05-02 02:20:57,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:20:57,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bc77e2fa3d7693ba7b97 HTTP/1.1" 200 None 2026-05-02 02:20:57,179 - lava_callback - DEBUG - Processing node 69f2bc77e2fa3d7693ba7b97: baseline-arm-clabbe 2026-05-02 02:20:57,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bc77e2fa3d7693ba7b97 2026-05-02 02:20:57,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:20:57,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bc77e2fa3d7693ba7b97 HTTP/1.1" 200 None 2026-05-02 02:20:57,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:20:57,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:20:57,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:20:57,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bc77e2fa3d7693ba7b97 HTTP/1.1" 200 None 2026-05-02 02:20:57,229 - lava_callback - ERROR - Error processing callback for node 69f2bc77e2fa3d7693ba7b97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58764 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:21:37,152 - lava_callback - INFO - Received callback for node 69f2bcace2fa3d7693ba7ba8 INFO: 172.20.0.1:58776 - "POST /node/69f2bcace2fa3d7693ba7ba8 HTTP/1.0" 202 Accepted 2026-05-02 02:21:37,157 - lava_callback - INFO - Starting async job submit for node 69f2bcace2fa3d7693ba7ba8 2026-05-02 02:21:37,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:21:37,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bcace2fa3d7693ba7ba8 HTTP/1.1" 200 None 2026-05-02 02:21:37,172 - lava_callback - DEBUG - Processing node 69f2bcace2fa3d7693ba7ba8: baseline-arm64-clabbe 2026-05-02 02:21:37,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bcace2fa3d7693ba7ba8 2026-05-02 02:21:37,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:21:37,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bcace2fa3d7693ba7ba8 HTTP/1.1" 200 None 2026-05-02 02:21:37,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:21:37,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:21:37,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:21:37,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bcace2fa3d7693ba7ba8 HTTP/1.1" 200 None 2026-05-02 02:21:37,224 - lava_callback - ERROR - Error processing callback for node 69f2bcace2fa3d7693ba7ba8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48330 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:22:57,275 - lava_callback - INFO - Received callback for node 69f2bcf9e2fa3d7693ba7bca INFO: 172.20.0.1:48336 - "POST /node/69f2bcf9e2fa3d7693ba7bca HTTP/1.0" 202 Accepted 2026-05-02 02:22:57,283 - lava_callback - INFO - Starting async job submit for node 69f2bcf9e2fa3d7693ba7bca 2026-05-02 02:22:57,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:22:57,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bcf9e2fa3d7693ba7bca HTTP/1.1" 200 None 2026-05-02 02:22:57,302 - lava_callback - DEBUG - Processing node 69f2bcf9e2fa3d7693ba7bca: baseline-arm64-clabbe 2026-05-02 02:22:57,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bcf9e2fa3d7693ba7bca 2026-05-02 02:22:57,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:22:57,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bcf9e2fa3d7693ba7bca HTTP/1.1" 200 None 2026-05-02 02:22:57,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:22:57,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 02:22:57,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:22:57,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bcf9e2fa3d7693ba7bca HTTP/1.1" 200 None 2026-05-02 02:22:57,348 - lava_callback - ERROR - Error processing callback for node 69f2bcf9e2fa3d7693ba7bca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39944 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:24:17,317 - lava_callback - INFO - Received callback for node 69f2bd48e2fa3d7693ba7bf0 INFO: 172.20.0.1:39948 - "POST /node/69f2bd48e2fa3d7693ba7bf0 HTTP/1.0" 202 Accepted 2026-05-02 02:24:17,323 - lava_callback - INFO - Starting async job submit for node 69f2bd48e2fa3d7693ba7bf0 2026-05-02 02:24:17,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:24:17,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bd48e2fa3d7693ba7bf0 HTTP/1.1" 200 None 2026-05-02 02:24:17,337 - lava_callback - DEBUG - Processing node 69f2bd48e2fa3d7693ba7bf0: baseline-arm64-clabbe 2026-05-02 02:24:17,337 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bd48e2fa3d7693ba7bf0 2026-05-02 02:24:17,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:24:17,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bd48e2fa3d7693ba7bf0 HTTP/1.1" 200 None 2026-05-02 02:24:17,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:24:17,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 02:24:17,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:24:17,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bd48e2fa3d7693ba7bf0 HTTP/1.1" 200 None 2026-05-02 02:24:17,399 - lava_callback - ERROR - Error processing callback for node 69f2bd48e2fa3d7693ba7bf0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:25:37,427 - lava_callback - INFO - Received callback for node 69f2bd9be2fa3d7693ba7c18 INFO: 172.20.0.1:45208 - "POST /node/69f2bd9be2fa3d7693ba7c18 HTTP/1.0" 202 Accepted 2026-05-02 02:25:37,433 - lava_callback - INFO - Starting async job submit for node 69f2bd9be2fa3d7693ba7c18 2026-05-02 02:25:37,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:25:37,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bd9be2fa3d7693ba7c18 HTTP/1.1" 200 None 2026-05-02 02:25:37,448 - lava_callback - DEBUG - Processing node 69f2bd9be2fa3d7693ba7c18: baseline-arm64-clabbe 2026-05-02 02:25:37,449 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bd9be2fa3d7693ba7c18 2026-05-02 02:25:37,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:25:37,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bd9be2fa3d7693ba7c18 HTTP/1.1" 200 None 2026-05-02 02:25:37,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:25:37,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 02:25:37,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:25:37,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bd9be2fa3d7693ba7c18 HTTP/1.1" 200 None 2026-05-02 02:25:37,507 - lava_callback - ERROR - Error processing callback for node 69f2bd9be2fa3d7693ba7c18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59884 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:26:34,256 - lava_callback - INFO - Received callback for node 69f54c0eed4ed40b022407c0 INFO: 172.20.0.1:53974 - "POST /node/69f54c0eed4ed40b022407c0 HTTP/1.0" 202 Accepted 2026-05-02 02:26:34,262 - lava_callback - INFO - Starting async job submit for node 69f54c0eed4ed40b022407c0 2026-05-02 02:26:34,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:34,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0eed4ed40b022407c0 HTTP/1.1" 200 None 2026-05-02 02:26:34,305 - lava_callback - DEBUG - Processing node 69f54c0eed4ed40b022407c0: baseline-arm64-broonie 2026-05-02 02:26:34,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:26:34,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:26:34,672 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0eed4ed40b022407c0/log.txt.gz 2026-05-02 02:26:34,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:26:35,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:26:35,093 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0eed4ed40b022407c0/lava-logs.yaml 2026-05-02 02:26:35,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:26:35,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:26:35,271 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0eed4ed40b022407c0/lava_callback.json.gz 2026-05-02 02:26:35,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:35,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0eed4ed40b022407c0 HTTP/1.1" 200 None 2026-05-02 02:26:35,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:35,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 02:26:35,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:35,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c0eed4ed40b022407c0 HTTP/1.1" 200 None 2026-05-02 02:26:35,353 - lava_callback - INFO - Completed processing callback for node 69f54c0eed4ed40b022407c0 INFO: 172.20.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47392 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:26:53,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:53,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-02 02:26:56,128 - lava_callback - INFO - Received callback for node 69f2bde4e2fa3d7693ba7c35 INFO: 172.20.0.1:43154 - "POST /node/69f2bde4e2fa3d7693ba7c35 HTTP/1.0" 202 Accepted 2026-05-02 02:26:56,133 - lava_callback - INFO - Starting async job submit for node 69f2bde4e2fa3d7693ba7c35 2026-05-02 02:26:56,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:56,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bde4e2fa3d7693ba7c35 HTTP/1.1" 200 None 2026-05-02 02:26:56,150 - lava_callback - DEBUG - Processing node 69f2bde4e2fa3d7693ba7c35: baseline-arm64-baylibre 2026-05-02 02:26:56,150 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bde4e2fa3d7693ba7c35 2026-05-02 02:26:56,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:56,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bde4e2fa3d7693ba7c35 HTTP/1.1" 200 None 2026-05-02 02:26:56,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:56,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:26:56,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:26:56,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bde4e2fa3d7693ba7c35 HTTP/1.1" 200 None 2026-05-02 02:26:56,207 - lava_callback - ERROR - Error processing callback for node 69f2bde4e2fa3d7693ba7c35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39220 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:27:57,530 - lava_callback - INFO - Received callback for node 69f2be2ae2fa3d7693ba7c5d INFO: 172.20.0.1:39232 - "POST /node/69f2be2ae2fa3d7693ba7c5d HTTP/1.0" 202 Accepted 2026-05-02 02:27:57,538 - lava_callback - INFO - Starting async job submit for node 69f2be2ae2fa3d7693ba7c5d 2026-05-02 02:27:57,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:27:57,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2be2ae2fa3d7693ba7c5d HTTP/1.1" 200 None 2026-05-02 02:27:57,555 - lava_callback - DEBUG - Processing node 69f2be2ae2fa3d7693ba7c5d: baseline-arm-clabbe 2026-05-02 02:27:57,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2be2ae2fa3d7693ba7c5d 2026-05-02 02:27:57,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:27:57,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2be2ae2fa3d7693ba7c5d HTTP/1.1" 200 None 2026-05-02 02:27:57,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:27:57,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:27:57,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:27:57,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2be2ae2fa3d7693ba7c5d HTTP/1.1" 200 None 2026-05-02 02:27:57,615 - lava_callback - ERROR - Error processing callback for node 69f2be2ae2fa3d7693ba7c5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54590 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:28:15,001 - lava_callback - INFO - Received callback for node 69f56134ed4ed40b0224134d INFO: 172.20.0.1:33448 - "POST /node/69f56134ed4ed40b0224134d HTTP/1.0" 202 Accepted 2026-05-02 02:28:15,007 - lava_callback - INFO - Starting async job submit for node 69f56134ed4ed40b0224134d 2026-05-02 02:28:15,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:15,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56134ed4ed40b0224134d HTTP/1.1" 200 None 2026-05-02 02:28:15,025 - lava_callback - DEBUG - Processing node 69f56134ed4ed40b0224134d: baseline-arm 2026-05-02 02:28:15,030 - lava_callback - WARNING - Failed to upload log 2026-05-02 02:28:15,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:28:15,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:28:15,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f56134ed4ed40b0224134d/lava-logs.yaml 2026-05-02 02:28:15,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33460 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 02:28:15,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:28:15,254 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f56134ed4ed40b0224134d/lava_callback.json.gz 2026-05-02 02:28:15,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:15,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56134ed4ed40b0224134d HTTP/1.1" 200 None 2026-05-02 02:28:15,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:15,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 02:28:15,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:15,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f56134ed4ed40b0224134d HTTP/1.1" 200 None 2026-05-02 02:28:15,326 - lava_callback - INFO - Completed processing callback for node 69f56134ed4ed40b0224134d INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60188 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:28:23,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:23,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:28:56,126 - lava_callback - INFO - Received callback for node 69f2be57e2fa3d7693ba7c6c INFO: 172.20.0.1:37594 - "POST /node/69f2be57e2fa3d7693ba7c6c HTTP/1.0" 202 Accepted 2026-05-02 02:28:56,132 - lava_callback - INFO - Starting async job submit for node 69f2be57e2fa3d7693ba7c6c 2026-05-02 02:28:56,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:56,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2be57e2fa3d7693ba7c6c HTTP/1.1" 200 None 2026-05-02 02:28:56,147 - lava_callback - DEBUG - Processing node 69f2be57e2fa3d7693ba7c6c: baseline-arm64-baylibre 2026-05-02 02:28:56,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2be57e2fa3d7693ba7c6c 2026-05-02 02:28:56,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:56,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2be57e2fa3d7693ba7c6c HTTP/1.1" 200 None 2026-05-02 02:28:56,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:56,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:28:56,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:28:56,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2be57e2fa3d7693ba7c6c HTTP/1.1" 200 None 2026-05-02 02:28:56,202 - lava_callback - ERROR - Error processing callback for node 69f2be57e2fa3d7693ba7c6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52956 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:29:57,673 - lava_callback - INFO - Received callback for node 69f2be9ee2fa3d7693ba7c91 INFO: 172.20.0.1:52962 - "POST /node/69f2be9ee2fa3d7693ba7c91 HTTP/1.0" 202 Accepted 2026-05-02 02:29:57,679 - lava_callback - INFO - Starting async job submit for node 69f2be9ee2fa3d7693ba7c91 2026-05-02 02:29:57,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:29:57,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2be9ee2fa3d7693ba7c91 HTTP/1.1" 200 None 2026-05-02 02:29:57,702 - lava_callback - DEBUG - Processing node 69f2be9ee2fa3d7693ba7c91: baseline-arm-clabbe 2026-05-02 02:29:57,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2be9ee2fa3d7693ba7c91 2026-05-02 02:29:57,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:29:57,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2be9ee2fa3d7693ba7c91 HTTP/1.1" 200 None 2026-05-02 02:29:57,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:29:57,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 02:29:57,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:29:57,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2be9ee2fa3d7693ba7c91 HTTP/1.1" 200 None 2026-05-02 02:29:57,749 - lava_callback - ERROR - Error processing callback for node 69f2be9ee2fa3d7693ba7c91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39734 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:30:37,628 - lava_callback - INFO - Received callback for node 69f2bec4e2fa3d7693ba7ca1 INFO: 172.20.0.1:39744 - "POST /node/69f2bec4e2fa3d7693ba7ca1 HTTP/1.0" 202 Accepted 2026-05-02 02:30:37,637 - lava_callback - INFO - Starting async job submit for node 69f2bec4e2fa3d7693ba7ca1 2026-05-02 02:30:37,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:30:37,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bec4e2fa3d7693ba7ca1 HTTP/1.1" 200 None 2026-05-02 02:30:37,654 - lava_callback - DEBUG - Processing node 69f2bec4e2fa3d7693ba7ca1: baseline-arm-clabbe 2026-05-02 02:30:37,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bec4e2fa3d7693ba7ca1 2026-05-02 02:30:37,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:30:37,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bec4e2fa3d7693ba7ca1 HTTP/1.1" 200 None 2026-05-02 02:30:37,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:30:37,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 02:30:37,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:30:37,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bec4e2fa3d7693ba7ca1 HTTP/1.1" 200 None 2026-05-02 02:30:37,715 - lava_callback - ERROR - Error processing callback for node 69f2bec4e2fa3d7693ba7ca1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36038 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:31:17,676 - lava_callback - INFO - Received callback for node 69f2beebe2fa3d7693ba7cb3 INFO: 172.20.0.1:36040 - "POST /node/69f2beebe2fa3d7693ba7cb3 HTTP/1.0" 202 Accepted 2026-05-02 02:31:17,681 - lava_callback - INFO - Starting async job submit for node 69f2beebe2fa3d7693ba7cb3 2026-05-02 02:31:17,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:17,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2beebe2fa3d7693ba7cb3 HTTP/1.1" 200 None 2026-05-02 02:31:17,696 - lava_callback - DEBUG - Processing node 69f2beebe2fa3d7693ba7cb3: baseline-arm-clabbe 2026-05-02 02:31:17,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2beebe2fa3d7693ba7cb3 2026-05-02 02:31:17,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:17,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2beebe2fa3d7693ba7cb3 HTTP/1.1" 200 None 2026-05-02 02:31:17,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:17,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 02:31:17,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:17,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2beebe2fa3d7693ba7cb3 HTTP/1.1" 200 None 2026-05-02 02:31:17,746 - lava_callback - ERROR - Error processing callback for node 69f2beebe2fa3d7693ba7cb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:31:54,912 - lava_callback - INFO - Received callback for node 69f55feded4ed40b0224129f INFO: 172.20.0.1:43292 - "POST /node/69f55feded4ed40b0224129f HTTP/1.0" 202 Accepted 2026-05-02 02:31:54,919 - lava_callback - INFO - Starting async job submit for node 69f55feded4ed40b0224129f 2026-05-02 02:31:54,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:54,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55feded4ed40b0224129f HTTP/1.1" 200 None 2026-05-02 02:31:54,957 - lava_callback - DEBUG - Processing node 69f55feded4ed40b0224129f: ltp-controllers 2026-05-02 02:31:55,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:31:55,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:31:55,499 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69f55feded4ed40b0224129f/log.txt.gz 2026-05-02 02:31:55,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:31:56,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:31:56,005 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69f55feded4ed40b0224129f/lava-logs.yaml 2026-05-02 02:31:56,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:31:56,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:31:56,144 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69f55feded4ed40b0224129f/lava_callback.json.gz 2026-05-02 02:31:56,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:56,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55feded4ed40b0224129f HTTP/1.1" 200 None 2026-05-02 02:31:56,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:56,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 02:31:56,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:56,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55feded4ed40b0224129f HTTP/1.1" 200 None 2026-05-02 02:31:56,240 - lava_callback - INFO - Completed processing callback for node 69f55feded4ed40b0224129f INFO: 172.20.0.1:43304 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:31:56,629 - lava_callback - INFO - Received callback for node 69f2bf16e2fa3d7693ba7cc8 INFO: 172.20.0.1:43308 - "POST /node/69f2bf16e2fa3d7693ba7cc8 HTTP/1.0" 202 Accepted 2026-05-02 02:31:56,634 - lava_callback - INFO - Starting async job submit for node 69f2bf16e2fa3d7693ba7cc8 2026-05-02 02:31:56,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:56,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bf16e2fa3d7693ba7cc8 HTTP/1.1" 200 None 2026-05-02 02:31:56,649 - lava_callback - DEBUG - Processing node 69f2bf16e2fa3d7693ba7cc8: baseline-arm64-baylibre 2026-05-02 02:31:56,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bf16e2fa3d7693ba7cc8 2026-05-02 02:31:56,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:56,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bf16e2fa3d7693ba7cc8 HTTP/1.1" 200 None 2026-05-02 02:31:56,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:56,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:31:56,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:31:56,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bf16e2fa3d7693ba7cc8 HTTP/1.1" 200 None 2026-05-02 02:31:56,696 - lava_callback - ERROR - Error processing callback for node 69f2bf16e2fa3d7693ba7cc8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54780 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:32:23,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:32:23,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:32:38,274 - lava_callback - INFO - Received callback for node 69f55906ed4ed40b02240f88 INFO: 172.20.0.1:42742 - "POST /node/69f55906ed4ed40b02240f88 HTTP/1.0" 202 Accepted 2026-05-02 02:32:38,280 - lava_callback - INFO - Starting async job submit for node 69f55906ed4ed40b02240f88 2026-05-02 02:32:38,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:32:38,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55906ed4ed40b02240f88 HTTP/1.1" 200 None 2026-05-02 02:32:38,312 - lava_callback - DEBUG - Processing node 69f55906ed4ed40b02240f88: baseline-arm64 2026-05-02 02:32:38,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:32:38,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:32:38,746 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55906ed4ed40b02240f88/log.txt.gz 2026-05-02 02:32:38,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:32:39,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:32:39,143 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55906ed4ed40b02240f88/lava-logs.yaml 2026-05-02 02:32:39,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:32:39,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:32:39,263 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55906ed4ed40b02240f88/lava_callback.json.gz 2026-05-02 02:32:39,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:32:39,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55906ed4ed40b02240f88 HTTP/1.1" 200 None 2026-05-02 02:32:39,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:32:39,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 02:32:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:32:39,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55906ed4ed40b02240f88 HTTP/1.1" 200 None 2026-05-02 02:32:39,388 - lava_callback - INFO - Completed processing callback for node 69f55906ed4ed40b02240f88 INFO: 172.20.0.1:58520 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:32:53,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:32:53,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:33:17,766 - lava_callback - INFO - Received callback for node 69f2bf5de2fa3d7693ba7cec INFO: 172.20.0.1:34574 - "POST /node/69f2bf5de2fa3d7693ba7cec HTTP/1.0" 202 Accepted 2026-05-02 02:33:17,771 - lava_callback - INFO - Starting async job submit for node 69f2bf5de2fa3d7693ba7cec 2026-05-02 02:33:17,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:17,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bf5de2fa3d7693ba7cec HTTP/1.1" 200 None 2026-05-02 02:33:17,787 - lava_callback - DEBUG - Processing node 69f2bf5de2fa3d7693ba7cec: baseline-arm-clabbe 2026-05-02 02:33:17,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bf5de2fa3d7693ba7cec 2026-05-02 02:33:17,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:17,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bf5de2fa3d7693ba7cec HTTP/1.1" 200 None 2026-05-02 02:33:17,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:17,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:33:17,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:17,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bf5de2fa3d7693ba7cec HTTP/1.1" 200 None 2026-05-02 02:33:17,833 - lava_callback - ERROR - Error processing callback for node 69f2bf5de2fa3d7693ba7cec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47820 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:33:56,831 - lava_callback - INFO - Received callback for node 69f2bf89e2fa3d7693ba7cfd INFO: 172.20.0.1:47824 - "POST /node/69f2bf89e2fa3d7693ba7cfd HTTP/1.0" 202 Accepted 2026-05-02 02:33:56,837 - lava_callback - INFO - Starting async job submit for node 69f2bf89e2fa3d7693ba7cfd 2026-05-02 02:33:56,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:56,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bf89e2fa3d7693ba7cfd HTTP/1.1" 200 None 2026-05-02 02:33:56,852 - lava_callback - DEBUG - Processing node 69f2bf89e2fa3d7693ba7cfd: baseline-arm64-baylibre 2026-05-02 02:33:56,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bf89e2fa3d7693ba7cfd 2026-05-02 02:33:56,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:56,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bf89e2fa3d7693ba7cfd HTTP/1.1" 200 None 2026-05-02 02:33:56,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:56,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:33:56,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:33:56,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bf89e2fa3d7693ba7cfd HTTP/1.1" 200 None 2026-05-02 02:33:56,904 - lava_callback - ERROR - Error processing callback for node 69f2bf89e2fa3d7693ba7cfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46016 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:34:58,110 - lava_callback - INFO - Received callback for node 69f2bfcfe2fa3d7693ba7d20 INFO: 172.20.0.1:46024 - "POST /node/69f2bfcfe2fa3d7693ba7d20 HTTP/1.0" 202 Accepted 2026-05-02 02:34:58,116 - lava_callback - INFO - Starting async job submit for node 69f2bfcfe2fa3d7693ba7d20 2026-05-02 02:34:58,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:34:58,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bfcfe2fa3d7693ba7d20 HTTP/1.1" 200 None 2026-05-02 02:34:58,151 - lava_callback - DEBUG - Processing node 69f2bfcfe2fa3d7693ba7d20: baseline-arm-clabbe 2026-05-02 02:34:58,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bfcfe2fa3d7693ba7d20 2026-05-02 02:34:58,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:34:58,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bfcfe2fa3d7693ba7d20 HTTP/1.1" 200 None 2026-05-02 02:34:58,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:34:58,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 02:34:58,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:34:58,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bfcfe2fa3d7693ba7d20 HTTP/1.1" 200 None 2026-05-02 02:34:58,205 - lava_callback - ERROR - Error processing callback for node 69f2bfcfe2fa3d7693ba7d20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46840 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:35:37,850 - lava_callback - INFO - Received callback for node 69f2bff3e2fa3d7693ba7d29 INFO: 172.20.0.1:46844 - "POST /node/69f2bff3e2fa3d7693ba7d29 HTTP/1.0" 202 Accepted 2026-05-02 02:35:37,856 - lava_callback - INFO - Starting async job submit for node 69f2bff3e2fa3d7693ba7d29 2026-05-02 02:35:37,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:35:37,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bff3e2fa3d7693ba7d29 HTTP/1.1" 200 None 2026-05-02 02:35:37,870 - lava_callback - DEBUG - Processing node 69f2bff3e2fa3d7693ba7d29: baseline-arm-clabbe 2026-05-02 02:35:37,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2bff3e2fa3d7693ba7d29 2026-05-02 02:35:37,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:35:37,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2bff3e2fa3d7693ba7d29 HTTP/1.1" 200 None 2026-05-02 02:35:37,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:35:37,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 02:35:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:35:37,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2bff3e2fa3d7693ba7d29 HTTP/1.1" 200 None 2026-05-02 02:35:37,917 - lava_callback - ERROR - Error processing callback for node 69f2bff3e2fa3d7693ba7d29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49322 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:36:17,912 - lava_callback - INFO - Received callback for node 69f2c018e2fa3d7693ba7d41 INFO: 172.20.0.1:49328 - "POST /node/69f2c018e2fa3d7693ba7d41 HTTP/1.0" 202 Accepted 2026-05-02 02:36:17,917 - lava_callback - INFO - Starting async job submit for node 69f2c018e2fa3d7693ba7d41 2026-05-02 02:36:17,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:17,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c018e2fa3d7693ba7d41 HTTP/1.1" 200 None 2026-05-02 02:36:17,933 - lava_callback - DEBUG - Processing node 69f2c018e2fa3d7693ba7d41: baseline-arm-clabbe 2026-05-02 02:36:17,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c018e2fa3d7693ba7d41 2026-05-02 02:36:17,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:17,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c018e2fa3d7693ba7d41 HTTP/1.1" 200 None 2026-05-02 02:36:17,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:17,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 02:36:17,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:17,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c018e2fa3d7693ba7d41 HTTP/1.1" 200 None 2026-05-02 02:36:17,982 - lava_callback - ERROR - Error processing callback for node 69f2c018e2fa3d7693ba7d41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35216 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:36:56,653 - lava_callback - INFO - Received callback for node 69f2c045e2fa3d7693ba7d56 INFO: 172.20.0.1:35218 - "POST /node/69f2c045e2fa3d7693ba7d56 HTTP/1.0" 202 Accepted 2026-05-02 02:36:56,659 - lava_callback - INFO - Starting async job submit for node 69f2c045e2fa3d7693ba7d56 2026-05-02 02:36:56,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:56,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c045e2fa3d7693ba7d56 HTTP/1.1" 200 None 2026-05-02 02:36:56,675 - lava_callback - DEBUG - Processing node 69f2c045e2fa3d7693ba7d56: baseline-arm64-baylibre 2026-05-02 02:36:56,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c045e2fa3d7693ba7d56 2026-05-02 02:36:56,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:56,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c045e2fa3d7693ba7d56 HTTP/1.1" 200 None 2026-05-02 02:36:56,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:56,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:36:56,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:36:56,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c045e2fa3d7693ba7d56 HTTP/1.1" 200 None 2026-05-02 02:36:56,722 - lava_callback - ERROR - Error processing callback for node 69f2c045e2fa3d7693ba7d56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:38:18,133 - lava_callback - INFO - Received callback for node 69f2c088e2fa3d7693ba7d73 INFO: 172.20.0.1:36258 - "POST /node/69f2c088e2fa3d7693ba7d73 HTTP/1.0" 202 Accepted 2026-05-02 02:38:18,138 - lava_callback - INFO - Starting async job submit for node 69f2c088e2fa3d7693ba7d73 2026-05-02 02:38:18,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:18,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c088e2fa3d7693ba7d73 HTTP/1.1" 200 None 2026-05-02 02:38:18,154 - lava_callback - DEBUG - Processing node 69f2c088e2fa3d7693ba7d73: baseline-arm-clabbe 2026-05-02 02:38:18,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c088e2fa3d7693ba7d73 2026-05-02 02:38:18,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:18,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c088e2fa3d7693ba7d73 HTTP/1.1" 200 None 2026-05-02 02:38:18,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:18,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:38:18,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:18,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c088e2fa3d7693ba7d73 HTTP/1.1" 200 None 2026-05-02 02:38:18,205 - lava_callback - ERROR - Error processing callback for node 69f2c088e2fa3d7693ba7d73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45332 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:38:56,731 - lava_callback - INFO - Received callback for node 69f2c0b5e2fa3d7693ba7d84 INFO: 172.20.0.1:45338 - "POST /node/69f2c0b5e2fa3d7693ba7d84 HTTP/1.0" 202 Accepted 2026-05-02 02:38:56,736 - lava_callback - INFO - Starting async job submit for node 69f2c0b5e2fa3d7693ba7d84 2026-05-02 02:38:56,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:56,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c0b5e2fa3d7693ba7d84 HTTP/1.1" 200 None 2026-05-02 02:38:56,771 - lava_callback - DEBUG - Processing node 69f2c0b5e2fa3d7693ba7d84: baseline-arm64-baylibre 2026-05-02 02:38:56,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c0b5e2fa3d7693ba7d84 2026-05-02 02:38:56,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:56,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c0b5e2fa3d7693ba7d84 HTTP/1.1" 200 None 2026-05-02 02:38:56,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:56,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:38:56,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:38:56,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c0b5e2fa3d7693ba7d84 HTTP/1.1" 200 None 2026-05-02 02:38:56,818 - lava_callback - ERROR - Error processing callback for node 69f2c0b5e2fa3d7693ba7d84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38184 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:41:18,274 - lava_callback - INFO - Received callback for node 69f2c14ae2fa3d7693ba7dc9 INFO: 172.20.0.1:38188 - "POST /node/69f2c14ae2fa3d7693ba7dc9 HTTP/1.0" 202 Accepted 2026-05-02 02:41:18,280 - lava_callback - INFO - Starting async job submit for node 69f2c14ae2fa3d7693ba7dc9 2026-05-02 02:41:18,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:18,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c14ae2fa3d7693ba7dc9 HTTP/1.1" 200 None 2026-05-02 02:41:18,296 - lava_callback - DEBUG - Processing node 69f2c14ae2fa3d7693ba7dc9: baseline-arm-clabbe 2026-05-02 02:41:18,296 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c14ae2fa3d7693ba7dc9 2026-05-02 02:41:18,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:18,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c14ae2fa3d7693ba7dc9 HTTP/1.1" 200 None 2026-05-02 02:41:18,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:18,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 02:41:18,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:18,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c14ae2fa3d7693ba7dc9 HTTP/1.1" 200 None 2026-05-02 02:41:18,350 - lava_callback - ERROR - Error processing callback for node 69f2c14ae2fa3d7693ba7dc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44758 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:41:58,212 - lava_callback - INFO - Received callback for node 69f2c16fe2fa3d7693ba7dde INFO: 172.20.0.1:44766 - "POST /node/69f2c16fe2fa3d7693ba7dde HTTP/1.0" 202 Accepted 2026-05-02 02:41:58,217 - lava_callback - INFO - Starting async job submit for node 69f2c16fe2fa3d7693ba7dde 2026-05-02 02:41:58,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:58,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c16fe2fa3d7693ba7dde HTTP/1.1" 200 None 2026-05-02 02:41:58,233 - lava_callback - DEBUG - Processing node 69f2c16fe2fa3d7693ba7dde: baseline-arm-clabbe 2026-05-02 02:41:58,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c16fe2fa3d7693ba7dde 2026-05-02 02:41:58,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:58,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c16fe2fa3d7693ba7dde HTTP/1.1" 200 None 2026-05-02 02:41:58,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:58,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 02:41:58,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:41:58,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c16fe2fa3d7693ba7dde HTTP/1.1" 200 None 2026-05-02 02:41:58,284 - lava_callback - ERROR - Error processing callback for node 69f2c16fe2fa3d7693ba7dde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34176 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:42:58,297 - lava_callback - INFO - Received callback for node 69f2c1a1e2fa3d7693ba7dec INFO: 172.20.0.1:34192 - "POST /node/69f2c1a1e2fa3d7693ba7dec HTTP/1.0" 202 Accepted 2026-05-02 02:42:58,302 - lava_callback - INFO - Starting async job submit for node 69f2c1a1e2fa3d7693ba7dec 2026-05-02 02:42:58,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:42:58,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c1a1e2fa3d7693ba7dec HTTP/1.1" 200 None 2026-05-02 02:42:58,318 - lava_callback - DEBUG - Processing node 69f2c1a1e2fa3d7693ba7dec: baseline-arm64-clabbe 2026-05-02 02:42:58,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c1a1e2fa3d7693ba7dec 2026-05-02 02:42:58,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:42:58,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c1a1e2fa3d7693ba7dec HTTP/1.1" 200 None 2026-05-02 02:42:58,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:42:58,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:42:58,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:42:58,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c1a1e2fa3d7693ba7dec HTTP/1.1" 200 None 2026-05-02 02:42:58,367 - lava_callback - ERROR - Error processing callback for node 69f2c1a1e2fa3d7693ba7dec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35410 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:43:58,342 - lava_callback - INFO - Received callback for node 69f2c1e1e2fa3d7693ba7e1d INFO: 172.20.0.1:35412 - "POST /node/69f2c1e1e2fa3d7693ba7e1d HTTP/1.0" 202 Accepted 2026-05-02 02:43:58,347 - lava_callback - INFO - Starting async job submit for node 69f2c1e1e2fa3d7693ba7e1d 2026-05-02 02:43:58,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:43:58,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c1e1e2fa3d7693ba7e1d HTTP/1.1" 200 None 2026-05-02 02:43:58,367 - lava_callback - DEBUG - Processing node 69f2c1e1e2fa3d7693ba7e1d: baseline-arm-clabbe 2026-05-02 02:43:58,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c1e1e2fa3d7693ba7e1d 2026-05-02 02:43:58,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:43:58,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c1e1e2fa3d7693ba7e1d HTTP/1.1" 200 None 2026-05-02 02:43:58,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:43:58,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 02:43:58,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:43:58,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c1e1e2fa3d7693ba7e1d HTTP/1.1" 200 None 2026-05-02 02:43:58,414 - lava_callback - ERROR - Error processing callback for node 69f2c1e1e2fa3d7693ba7e1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59878 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:44:38,296 - lava_callback - INFO - Received callback for node 69f2c206e2fa3d7693ba7e2f INFO: 172.20.0.1:59894 - "POST /node/69f2c206e2fa3d7693ba7e2f HTTP/1.0" 202 Accepted 2026-05-02 02:44:38,301 - lava_callback - INFO - Starting async job submit for node 69f2c206e2fa3d7693ba7e2f 2026-05-02 02:44:38,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:44:38,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c206e2fa3d7693ba7e2f HTTP/1.1" 200 None 2026-05-02 02:44:38,320 - lava_callback - DEBUG - Processing node 69f2c206e2fa3d7693ba7e2f: baseline-arm-clabbe 2026-05-02 02:44:38,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c206e2fa3d7693ba7e2f 2026-05-02 02:44:38,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:44:38,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c206e2fa3d7693ba7e2f HTTP/1.1" 200 None 2026-05-02 02:44:38,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:44:38,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:44:38,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:44:38,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c206e2fa3d7693ba7e2f HTTP/1.1" 200 None 2026-05-02 02:44:38,366 - lava_callback - ERROR - Error processing callback for node 69f2c206e2fa3d7693ba7e2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:45:18,355 - lava_callback - INFO - Received callback for node 69f2c23ae2fa3d7693ba7e41 INFO: 172.20.0.1:59992 - "POST /node/69f2c23ae2fa3d7693ba7e41 HTTP/1.0" 202 Accepted 2026-05-02 02:45:18,362 - lava_callback - INFO - Starting async job submit for node 69f2c23ae2fa3d7693ba7e41 2026-05-02 02:45:18,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:45:18,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c23ae2fa3d7693ba7e41 HTTP/1.1" 200 None 2026-05-02 02:45:18,382 - lava_callback - DEBUG - Processing node 69f2c23ae2fa3d7693ba7e41: baseline-arm64-clabbe 2026-05-02 02:45:18,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c23ae2fa3d7693ba7e41 2026-05-02 02:45:18,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:45:18,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c23ae2fa3d7693ba7e41 HTTP/1.1" 200 None 2026-05-02 02:45:18,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:45:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:45:18,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:45:18,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c23ae2fa3d7693ba7e41 HTTP/1.1" 200 None 2026-05-02 02:45:18,435 - lava_callback - ERROR - Error processing callback for node 69f2c23ae2fa3d7693ba7e41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42770 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:46:38,421 - lava_callback - INFO - Received callback for node 69f2c27ee2fa3d7693ba7e65 INFO: 172.20.0.1:42782 - "POST /node/69f2c27ee2fa3d7693ba7e65 HTTP/1.0" 202 Accepted 2026-05-02 02:46:38,427 - lava_callback - INFO - Starting async job submit for node 69f2c27ee2fa3d7693ba7e65 2026-05-02 02:46:38,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:46:38,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c27ee2fa3d7693ba7e65 HTTP/1.1" 200 None 2026-05-02 02:46:38,445 - lava_callback - DEBUG - Processing node 69f2c27ee2fa3d7693ba7e65: baseline-arm-clabbe 2026-05-02 02:46:38,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c27ee2fa3d7693ba7e65 2026-05-02 02:46:38,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:46:38,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c27ee2fa3d7693ba7e65 HTTP/1.1" 200 None 2026-05-02 02:46:38,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:46:38,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 02:46:38,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:46:38,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c27ee2fa3d7693ba7e65 HTTP/1.1" 200 None 2026-05-02 02:46:38,493 - lava_callback - ERROR - Error processing callback for node 69f2c27ee2fa3d7693ba7e65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:47:18,432 - lava_callback - INFO - Received callback for node 69f2c2a4e2fa3d7693ba7e74 INFO: 172.20.0.1:50092 - "POST /node/69f2c2a4e2fa3d7693ba7e74 HTTP/1.0" 202 Accepted 2026-05-02 02:47:18,437 - lava_callback - INFO - Starting async job submit for node 69f2c2a4e2fa3d7693ba7e74 2026-05-02 02:47:18,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:18,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c2a4e2fa3d7693ba7e74 HTTP/1.1" 200 None 2026-05-02 02:47:18,451 - lava_callback - DEBUG - Processing node 69f2c2a4e2fa3d7693ba7e74: baseline-arm-clabbe 2026-05-02 02:47:18,452 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c2a4e2fa3d7693ba7e74 2026-05-02 02:47:18,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:18,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c2a4e2fa3d7693ba7e74 HTTP/1.1" 200 None 2026-05-02 02:47:18,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:18,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 02:47:18,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:18,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c2a4e2fa3d7693ba7e74 HTTP/1.1" 200 None 2026-05-02 02:47:18,519 - lava_callback - ERROR - Error processing callback for node 69f2c2a4e2fa3d7693ba7e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34908 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:47:58,511 - lava_callback - INFO - Received callback for node 69f2c2d8e2fa3d7693ba7e81 INFO: 172.20.0.1:34912 - "POST /node/69f2c2d8e2fa3d7693ba7e81 HTTP/1.0" 202 Accepted 2026-05-02 02:47:58,516 - lava_callback - INFO - Starting async job submit for node 69f2c2d8e2fa3d7693ba7e81 2026-05-02 02:47:58,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:58,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c2d8e2fa3d7693ba7e81 HTTP/1.1" 200 None 2026-05-02 02:47:58,531 - lava_callback - DEBUG - Processing node 69f2c2d8e2fa3d7693ba7e81: baseline-arm64-clabbe 2026-05-02 02:47:58,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c2d8e2fa3d7693ba7e81 2026-05-02 02:47:58,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:58,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c2d8e2fa3d7693ba7e81 HTTP/1.1" 200 None 2026-05-02 02:47:58,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:58,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:47:58,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:47:58,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c2d8e2fa3d7693ba7e81 HTTP/1.1" 200 None 2026-05-02 02:47:58,575 - lava_callback - ERROR - Error processing callback for node 69f2c2d8e2fa3d7693ba7e81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55956 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:48:26,806 - lava_callback - INFO - Received callback for node 69f5583aed4ed40b02240f0b INFO: 172.20.0.1:55968 - "POST /node/69f5583aed4ed40b02240f0b HTTP/1.0" 202 Accepted 2026-05-02 02:48:26,812 - lava_callback - INFO - Starting async job submit for node 69f5583aed4ed40b02240f0b 2026-05-02 02:48:26,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:48:26,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5583aed4ed40b02240f0b HTTP/1.1" 200 None 2026-05-02 02:48:26,849 - lava_callback - DEBUG - Processing node 69f5583aed4ed40b02240f0b: baseline-arm64 2026-05-02 02:48:27,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:48:27,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:48:27,282 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f5583aed4ed40b02240f0b/log.txt.gz 2026-05-02 02:48:27,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:48:27,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:48:27,572 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f5583aed4ed40b02240f0b/lava-logs.yaml 2026-05-02 02:48:27,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:48:27,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:48:27,686 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f5583aed4ed40b02240f0b/lava_callback.json.gz 2026-05-02 02:48:27,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:48:27,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5583aed4ed40b02240f0b HTTP/1.1" 200 None 2026-05-02 02:48:27,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:48:27,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 02:48:27,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:48:27,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f5583aed4ed40b02240f0b HTTP/1.1" 200 None 2026-05-02 02:48:27,768 - lava_callback - INFO - Completed processing callback for node 69f5583aed4ed40b02240f0b INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53996 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:48:53,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:48:53,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40562 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:49:18,528 - lava_callback - INFO - Received callback for node 69f2c31ae2fa3d7693ba7ea6 INFO: 172.20.0.1:40564 - "POST /node/69f2c31ae2fa3d7693ba7ea6 HTTP/1.0" 202 Accepted 2026-05-02 02:49:18,533 - lava_callback - INFO - Starting async job submit for node 69f2c31ae2fa3d7693ba7ea6 2026-05-02 02:49:18,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:18,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c31ae2fa3d7693ba7ea6 HTTP/1.1" 200 None 2026-05-02 02:49:18,565 - lava_callback - DEBUG - Processing node 69f2c31ae2fa3d7693ba7ea6: baseline-arm-clabbe 2026-05-02 02:49:18,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c31ae2fa3d7693ba7ea6 2026-05-02 02:49:18,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:18,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c31ae2fa3d7693ba7ea6 HTTP/1.1" 200 None 2026-05-02 02:49:18,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:18,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 02:49:18,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:18,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c31ae2fa3d7693ba7ea6 HTTP/1.1" 200 None 2026-05-02 02:49:18,612 - lava_callback - ERROR - Error processing callback for node 69f2c31ae2fa3d7693ba7ea6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59502 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:49:38,536 - lava_callback - INFO - Received callback for node 69f2c33fe2fa3d7693ba7eb1 INFO: 172.20.0.1:59516 - "POST /node/69f2c33fe2fa3d7693ba7eb1 HTTP/1.0" 202 Accepted 2026-05-02 02:49:38,542 - lava_callback - INFO - Starting async job submit for node 69f2c33fe2fa3d7693ba7eb1 2026-05-02 02:49:38,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:38,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c33fe2fa3d7693ba7eb1 HTTP/1.1" 200 None 2026-05-02 02:49:38,566 - lava_callback - DEBUG - Processing node 69f2c33fe2fa3d7693ba7eb1: baseline-arm-clabbe 2026-05-02 02:49:38,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c33fe2fa3d7693ba7eb1 2026-05-02 02:49:38,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:38,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c33fe2fa3d7693ba7eb1 HTTP/1.1" 200 None 2026-05-02 02:49:38,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:38,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:49:38,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:49:38,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c33fe2fa3d7693ba7eb1 HTTP/1.1" 200 None 2026-05-02 02:49:38,610 - lava_callback - ERROR - Error processing callback for node 69f2c33fe2fa3d7693ba7eb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33720 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:50:38,588 - lava_callback - INFO - Received callback for node 69f2c374e2fa3d7693ba7ec5 INFO: 172.20.0.1:33728 - "POST /node/69f2c374e2fa3d7693ba7ec5 HTTP/1.0" 202 Accepted 2026-05-02 02:50:38,594 - lava_callback - INFO - Starting async job submit for node 69f2c374e2fa3d7693ba7ec5 2026-05-02 02:50:38,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:50:38,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c374e2fa3d7693ba7ec5 HTTP/1.1" 200 None 2026-05-02 02:50:38,619 - lava_callback - DEBUG - Processing node 69f2c374e2fa3d7693ba7ec5: baseline-arm64-clabbe 2026-05-02 02:50:38,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c374e2fa3d7693ba7ec5 2026-05-02 02:50:38,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:50:38,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c374e2fa3d7693ba7ec5 HTTP/1.1" 200 None 2026-05-02 02:50:38,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:50:38,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:50:38,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:50:38,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c374e2fa3d7693ba7ec5 HTTP/1.1" 200 None 2026-05-02 02:50:38,669 - lava_callback - ERROR - Error processing callback for node 69f2c374e2fa3d7693ba7ec5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40070 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:51:38,659 - lava_callback - INFO - Received callback for node 69f2c3b4e2fa3d7693ba7ee2 INFO: 172.20.0.1:40078 - "POST /node/69f2c3b4e2fa3d7693ba7ee2 HTTP/1.0" 202 Accepted 2026-05-02 02:51:38,664 - lava_callback - INFO - Starting async job submit for node 69f2c3b4e2fa3d7693ba7ee2 2026-05-02 02:51:38,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:51:38,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c3b4e2fa3d7693ba7ee2 HTTP/1.1" 200 None 2026-05-02 02:51:38,681 - lava_callback - DEBUG - Processing node 69f2c3b4e2fa3d7693ba7ee2: baseline-arm-clabbe 2026-05-02 02:51:38,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c3b4e2fa3d7693ba7ee2 2026-05-02 02:51:38,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:51:38,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c3b4e2fa3d7693ba7ee2 HTTP/1.1" 200 None 2026-05-02 02:51:38,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:51:38,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 02:51:38,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:51:38,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c3b4e2fa3d7693ba7ee2 HTTP/1.1" 200 None 2026-05-02 02:51:38,732 - lava_callback - ERROR - Error processing callback for node 69f2c3b4e2fa3d7693ba7ee2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36368 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:52:18,666 - lava_callback - INFO - Received callback for node 69f2c3dae2fa3d7693ba7ef0 INFO: 172.20.0.1:36378 - "POST /node/69f2c3dae2fa3d7693ba7ef0 HTTP/1.0" 202 Accepted 2026-05-02 02:52:18,672 - lava_callback - INFO - Starting async job submit for node 69f2c3dae2fa3d7693ba7ef0 2026-05-02 02:52:18,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:52:18,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c3dae2fa3d7693ba7ef0 HTTP/1.1" 200 None 2026-05-02 02:52:18,696 - lava_callback - DEBUG - Processing node 69f2c3dae2fa3d7693ba7ef0: baseline-arm-clabbe 2026-05-02 02:52:18,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c3dae2fa3d7693ba7ef0 2026-05-02 02:52:18,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:52:18,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c3dae2fa3d7693ba7ef0 HTTP/1.1" 200 None 2026-05-02 02:52:18,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:52:18,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 02:52:18,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:52:18,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c3dae2fa3d7693ba7ef0 HTTP/1.1" 200 None 2026-05-02 02:52:18,780 - lava_callback - ERROR - Error processing callback for node 69f2c3dae2fa3d7693ba7ef0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41886 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:53:18,789 - lava_callback - INFO - Received callback for node 69f2c40de2fa3d7693ba7eff INFO: 172.20.0.1:41902 - "POST /node/69f2c40de2fa3d7693ba7eff HTTP/1.0" 202 Accepted 2026-05-02 02:53:18,794 - lava_callback - INFO - Starting async job submit for node 69f2c40de2fa3d7693ba7eff 2026-05-02 02:53:18,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:53:18,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c40de2fa3d7693ba7eff HTTP/1.1" 200 None 2026-05-02 02:53:18,811 - lava_callback - DEBUG - Processing node 69f2c40de2fa3d7693ba7eff: baseline-arm64-clabbe 2026-05-02 02:53:18,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c40de2fa3d7693ba7eff 2026-05-02 02:53:18,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:53:18,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c40de2fa3d7693ba7eff HTTP/1.1" 200 None 2026-05-02 02:53:18,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:53:18,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:53:18,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:53:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c40de2fa3d7693ba7eff HTTP/1.1" 200 None 2026-05-02 02:53:18,865 - lava_callback - ERROR - Error processing callback for node 69f2c40de2fa3d7693ba7eff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:54:18,812 - lava_callback - INFO - Received callback for node 69f2c44ae2fa3d7693ba7f20 INFO: 172.20.0.1:36610 - "POST /node/69f2c44ae2fa3d7693ba7f20 HTTP/1.0" 202 Accepted 2026-05-02 02:54:18,818 - lava_callback - INFO - Starting async job submit for node 69f2c44ae2fa3d7693ba7f20 2026-05-02 02:54:18,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:18,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c44ae2fa3d7693ba7f20 HTTP/1.1" 200 None 2026-05-02 02:54:18,832 - lava_callback - DEBUG - Processing node 69f2c44ae2fa3d7693ba7f20: baseline-arm-clabbe 2026-05-02 02:54:18,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c44ae2fa3d7693ba7f20 2026-05-02 02:54:18,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:18,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c44ae2fa3d7693ba7f20 HTTP/1.1" 200 None 2026-05-02 02:54:18,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:18,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 02:54:18,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:18,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c44ae2fa3d7693ba7f20 HTTP/1.1" 200 None 2026-05-02 02:54:18,886 - lava_callback - ERROR - Error processing callback for node 69f2c44ae2fa3d7693ba7f20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42248 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:54:44,934 - lava_callback - INFO - Received callback for node 69f562dbed4ed40b02241431 INFO: 172.20.0.1:51056 - "POST /node/69f562dbed4ed40b02241431 HTTP/1.0" 202 Accepted 2026-05-02 02:54:44,943 - lava_callback - INFO - Starting async job submit for node 69f562dbed4ed40b02241431 2026-05-02 02:54:44,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:44,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f562dbed4ed40b02241431 HTTP/1.1" 200 None 2026-05-02 02:54:44,999 - lava_callback - DEBUG - Processing node 69f562dbed4ed40b02241431: baseline-nfs-arm64 2026-05-02 02:54:46,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:54:46,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:54:46,330 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f562dbed4ed40b02241431/log.txt.gz INFO: 172.20.0.1:51070 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:54:46,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:54:47,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:54:47,111 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f562dbed4ed40b02241431/lava-logs.yaml 2026-05-02 02:54:47,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 02:54:47,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 02:54:47,267 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f562dbed4ed40b02241431/lava_callback.json.gz DEBUG: Setup failed for 69f562dbed4ed40b02241431. Transitting job node result: incomplete -> fail 2026-05-02 02:54:47,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:47,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f562dbed4ed40b02241431 HTTP/1.1" 200 None 2026-05-02 02:54:47,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:47,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 02:54:47,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:47,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f562dbed4ed40b02241431 HTTP/1.1" 200 None 2026-05-02 02:54:47,497 - lava_callback - INFO - Completed processing callback for node 69f562dbed4ed40b02241431 2026-05-02 02:54:53,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:53,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33430 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:54:58,796 - lava_callback - INFO - Received callback for node 69f2c47be2fa3d7693ba7f32 INFO: 172.20.0.1:33434 - "POST /node/69f2c47be2fa3d7693ba7f32 HTTP/1.0" 202 Accepted 2026-05-02 02:54:58,804 - lava_callback - INFO - Starting async job submit for node 69f2c47be2fa3d7693ba7f32 2026-05-02 02:54:58,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:58,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c47be2fa3d7693ba7f32 HTTP/1.1" 200 None 2026-05-02 02:54:58,819 - lava_callback - DEBUG - Processing node 69f2c47be2fa3d7693ba7f32: baseline-arm64-clabbe 2026-05-02 02:54:58,820 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c47be2fa3d7693ba7f32 2026-05-02 02:54:58,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:58,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c47be2fa3d7693ba7f32 HTTP/1.1" 200 None 2026-05-02 02:54:58,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:58,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:54:58,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:54:58,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c47be2fa3d7693ba7f32 HTTP/1.1" 200 None 2026-05-02 02:54:58,895 - lava_callback - ERROR - Error processing callback for node 69f2c47be2fa3d7693ba7f32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50260 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:55:58,878 - lava_callback - INFO - Received callback for node 69f2c4b8e2fa3d7693ba7f55 INFO: 172.20.0.1:50274 - "POST /node/69f2c4b8e2fa3d7693ba7f55 HTTP/1.0" 202 Accepted 2026-05-02 02:55:58,884 - lava_callback - INFO - Starting async job submit for node 69f2c4b8e2fa3d7693ba7f55 2026-05-02 02:55:58,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:55:58,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c4b8e2fa3d7693ba7f55 HTTP/1.1" 200 None 2026-05-02 02:55:58,899 - lava_callback - DEBUG - Processing node 69f2c4b8e2fa3d7693ba7f55: baseline-arm-clabbe 2026-05-02 02:55:58,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c4b8e2fa3d7693ba7f55 2026-05-02 02:55:58,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:55:58,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c4b8e2fa3d7693ba7f55 HTTP/1.1" 200 None 2026-05-02 02:55:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:55:58,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:55:58,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:55:58,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c4b8e2fa3d7693ba7f55 HTTP/1.1" 200 None 2026-05-02 02:55:58,947 - lava_callback - ERROR - Error processing callback for node 69f2c4b8e2fa3d7693ba7f55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:56:38,847 - lava_callback - INFO - Received callback for node 69f2c4dde2fa3d7693ba7f63 INFO: 172.20.0.1:37892 - "POST /node/69f2c4dde2fa3d7693ba7f63 HTTP/1.0" 202 Accepted 2026-05-02 02:56:38,852 - lava_callback - INFO - Starting async job submit for node 69f2c4dde2fa3d7693ba7f63 2026-05-02 02:56:38,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:56:38,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c4dde2fa3d7693ba7f63 HTTP/1.1" 200 None 2026-05-02 02:56:38,872 - lava_callback - DEBUG - Processing node 69f2c4dde2fa3d7693ba7f63: baseline-arm-clabbe 2026-05-02 02:56:38,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c4dde2fa3d7693ba7f63 2026-05-02 02:56:38,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:56:38,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c4dde2fa3d7693ba7f63 HTTP/1.1" 200 None 2026-05-02 02:56:38,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:56:38,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 02:56:38,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:56:38,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c4dde2fa3d7693ba7f63 HTTP/1.1" 200 None 2026-05-02 02:56:38,916 - lava_callback - ERROR - Error processing callback for node 69f2c4dde2fa3d7693ba7f63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37182 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:57:38,917 - lava_callback - INFO - Received callback for node 69f2c50fe2fa3d7693ba7f72 INFO: 172.20.0.1:37188 - "POST /node/69f2c50fe2fa3d7693ba7f72 HTTP/1.0" 202 Accepted 2026-05-02 02:57:38,922 - lava_callback - INFO - Starting async job submit for node 69f2c50fe2fa3d7693ba7f72 2026-05-02 02:57:38,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:57:38,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c50fe2fa3d7693ba7f72 HTTP/1.1" 200 None 2026-05-02 02:57:38,939 - lava_callback - DEBUG - Processing node 69f2c50fe2fa3d7693ba7f72: baseline-arm64-clabbe 2026-05-02 02:57:38,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c50fe2fa3d7693ba7f72 2026-05-02 02:57:38,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:57:38,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c50fe2fa3d7693ba7f72 HTTP/1.1" 200 None 2026-05-02 02:57:38,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:57:38,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 02:57:38,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:57:38,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c50fe2fa3d7693ba7f72 HTTP/1.1" 200 None 2026-05-02 02:57:38,998 - lava_callback - ERROR - Error processing callback for node 69f2c50fe2fa3d7693ba7f72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45448 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:58:38,985 - lava_callback - INFO - Received callback for node 69f2c54ee2fa3d7693ba7f91 INFO: 172.20.0.1:45452 - "POST /node/69f2c54ee2fa3d7693ba7f91 HTTP/1.0" 202 Accepted 2026-05-02 02:58:38,991 - lava_callback - INFO - Starting async job submit for node 69f2c54ee2fa3d7693ba7f91 2026-05-02 02:58:38,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:58:39,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c54ee2fa3d7693ba7f91 HTTP/1.1" 200 None 2026-05-02 02:58:39,005 - lava_callback - DEBUG - Processing node 69f2c54ee2fa3d7693ba7f91: baseline-arm-clabbe 2026-05-02 02:58:39,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c54ee2fa3d7693ba7f91 2026-05-02 02:58:39,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:58:39,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c54ee2fa3d7693ba7f91 HTTP/1.1" 200 None 2026-05-02 02:58:39,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:58:39,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 02:58:39,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:58:39,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c54ee2fa3d7693ba7f91 HTTP/1.1" 200 None 2026-05-02 02:58:39,050 - lava_callback - ERROR - Error processing callback for node 69f2c54ee2fa3d7693ba7f91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60114 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:59:18,958 - lava_callback - INFO - Received callback for node 69f2c572e2fa3d7693ba7fa3 INFO: 172.20.0.1:60116 - "POST /node/69f2c572e2fa3d7693ba7fa3 HTTP/1.0" 202 Accepted 2026-05-02 02:59:18,963 - lava_callback - INFO - Starting async job submit for node 69f2c572e2fa3d7693ba7fa3 2026-05-02 02:59:18,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:18,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c572e2fa3d7693ba7fa3 HTTP/1.1" 200 None 2026-05-02 02:59:18,987 - lava_callback - DEBUG - Processing node 69f2c572e2fa3d7693ba7fa3: baseline-arm-clabbe 2026-05-02 02:59:18,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c572e2fa3d7693ba7fa3 2026-05-02 02:59:18,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:19,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c572e2fa3d7693ba7fa3 HTTP/1.1" 200 None 2026-05-02 02:59:19,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:19,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 02:59:19,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:19,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c572e2fa3d7693ba7fa3 HTTP/1.1" 200 None 2026-05-02 02:59:19,034 - lava_callback - ERROR - Error processing callback for node 69f2c572e2fa3d7693ba7fa3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK 2026-05-02 02:59:39,014 - lava_callback - INFO - Received callback for node 69f2c596e2fa3d7693ba7fae INFO: 172.20.0.1:45054 - "POST /node/69f2c596e2fa3d7693ba7fae HTTP/1.0" 202 Accepted 2026-05-02 02:59:39,020 - lava_callback - INFO - Starting async job submit for node 69f2c596e2fa3d7693ba7fae 2026-05-02 02:59:39,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:39,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c596e2fa3d7693ba7fae HTTP/1.1" 200 None 2026-05-02 02:59:39,035 - lava_callback - DEBUG - Processing node 69f2c596e2fa3d7693ba7fae: baseline-arm-clabbe 2026-05-02 02:59:39,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c596e2fa3d7693ba7fae 2026-05-02 02:59:39,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:39,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c596e2fa3d7693ba7fae HTTP/1.1" 200 None 2026-05-02 02:59:39,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:39,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 02:59:39,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 02:59:39,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c596e2fa3d7693ba7fae HTTP/1.1" 200 None 2026-05-02 02:59:39,084 - lava_callback - ERROR - Error processing callback for node 69f2c596e2fa3d7693ba7fae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60126 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:00:18,930 - lava_callback - INFO - Received callback for node 69f2c5bae2fa3d7693ba7fbe INFO: 172.20.0.1:60142 - "POST /node/69f2c5bae2fa3d7693ba7fbe HTTP/1.0" 202 Accepted 2026-05-02 03:00:18,936 - lava_callback - INFO - Starting async job submit for node 69f2c5bae2fa3d7693ba7fbe 2026-05-02 03:00:18,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:00:18,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c5bae2fa3d7693ba7fbe HTTP/1.1" 200 None 2026-05-02 03:00:18,961 - lava_callback - DEBUG - Processing node 69f2c5bae2fa3d7693ba7fbe: baseline-arm-clabbe 2026-05-02 03:00:18,961 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c5bae2fa3d7693ba7fbe 2026-05-02 03:00:18,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:00:18,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c5bae2fa3d7693ba7fbe HTTP/1.1" 200 None 2026-05-02 03:00:18,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:00:19,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:00:19,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:00:19,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c5bae2fa3d7693ba7fbe HTTP/1.1" 200 None 2026-05-02 03:00:19,027 - lava_callback - ERROR - Error processing callback for node 69f2c5bae2fa3d7693ba7fbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46728 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:01:39,155 - lava_callback - INFO - Received callback for node 69f2c604e2fa3d7693ba7fdc INFO: 172.20.0.1:46730 - "POST /node/69f2c604e2fa3d7693ba7fdc HTTP/1.0" 202 Accepted 2026-05-02 03:01:39,164 - lava_callback - INFO - Starting async job submit for node 69f2c604e2fa3d7693ba7fdc 2026-05-02 03:01:39,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:01:39,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c604e2fa3d7693ba7fdc HTTP/1.1" 200 None 2026-05-02 03:01:39,182 - lava_callback - DEBUG - Processing node 69f2c604e2fa3d7693ba7fdc: baseline-arm-clabbe 2026-05-02 03:01:39,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c604e2fa3d7693ba7fdc 2026-05-02 03:01:39,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:01:39,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c604e2fa3d7693ba7fdc HTTP/1.1" 200 None 2026-05-02 03:01:39,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:01:39,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 03:01:39,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:01:39,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c604e2fa3d7693ba7fdc HTTP/1.1" 200 None 2026-05-02 03:01:39,228 - lava_callback - ERROR - Error processing callback for node 69f2c604e2fa3d7693ba7fdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38810 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:02:19,093 - lava_callback - INFO - Received callback for node 69f2c628e2fa3d7693ba7fe8 INFO: 172.20.0.1:38816 - "POST /node/69f2c628e2fa3d7693ba7fe8 HTTP/1.0" 202 Accepted 2026-05-02 03:02:19,098 - lava_callback - INFO - Starting async job submit for node 69f2c628e2fa3d7693ba7fe8 2026-05-02 03:02:19,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:19,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c628e2fa3d7693ba7fe8 HTTP/1.1" 200 None 2026-05-02 03:02:19,114 - lava_callback - DEBUG - Processing node 69f2c628e2fa3d7693ba7fe8: baseline-arm-clabbe 2026-05-02 03:02:19,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c628e2fa3d7693ba7fe8 2026-05-02 03:02:19,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:19,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c628e2fa3d7693ba7fe8 HTTP/1.1" 200 None 2026-05-02 03:02:19,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:19,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 03:02:19,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:19,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c628e2fa3d7693ba7fe8 HTTP/1.1" 200 None 2026-05-02 03:02:19,175 - lava_callback - ERROR - Error processing callback for node 69f2c628e2fa3d7693ba7fe8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54950 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:02:59,118 - lava_callback - INFO - Received callback for node 69f2c64de2fa3d7693ba7ff7 INFO: 172.20.0.1:54954 - "POST /node/69f2c64de2fa3d7693ba7ff7 HTTP/1.0" 202 Accepted 2026-05-02 03:02:59,124 - lava_callback - INFO - Starting async job submit for node 69f2c64de2fa3d7693ba7ff7 2026-05-02 03:02:59,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:59,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c64de2fa3d7693ba7ff7 HTTP/1.1" 200 None 2026-05-02 03:02:59,138 - lava_callback - DEBUG - Processing node 69f2c64de2fa3d7693ba7ff7: baseline-arm-clabbe 2026-05-02 03:02:59,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c64de2fa3d7693ba7ff7 2026-05-02 03:02:59,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:59,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c64de2fa3d7693ba7ff7 HTTP/1.1" 200 None 2026-05-02 03:02:59,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:59,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:02:59,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:02:59,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c64de2fa3d7693ba7ff7 HTTP/1.1" 200 None 2026-05-02 03:02:59,184 - lava_callback - ERROR - Error processing callback for node 69f2c64de2fa3d7693ba7ff7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:03:29,979 - lava_callback - INFO - Received callback for node 69f54c0bed4ed40b022407b4 INFO: 172.20.0.1:44994 - "POST /node/69f54c0bed4ed40b022407b4 HTTP/1.0" 202 Accepted 2026-05-02 03:03:29,985 - lava_callback - INFO - Starting async job submit for node 69f54c0bed4ed40b022407b4 2026-05-02 03:03:30,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:30,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0bed4ed40b022407b4 HTTP/1.1" 200 None 2026-05-02 03:03:30,022 - lava_callback - DEBUG - Processing node 69f54c0bed4ed40b022407b4: baseline-arm64-broonie 2026-05-02 03:03:30,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:03:30,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:03:30,362 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0bed4ed40b022407b4/log.txt.gz 2026-05-02 03:03:30,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:03:30,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:03:30,651 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0bed4ed40b022407b4/lava-logs.yaml 2026-05-02 03:03:30,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:03:30,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:03:30,757 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f54c0bed4ed40b022407b4/lava_callback.json.gz 2026-05-02 03:03:30,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:30,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54c0bed4ed40b022407b4 HTTP/1.1" 200 None 2026-05-02 03:03:30,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:30,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 03:03:30,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:30,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f54c0bed4ed40b022407b4 HTTP/1.1" 200 None 2026-05-02 03:03:30,830 - lava_callback - INFO - Completed processing callback for node 69f54c0bed4ed40b022407b4 INFO: 172.20.0.1:45006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40956 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:03:53,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:53,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53450 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:03:59,160 - lava_callback - INFO - Received callback for node 69f2c696e2fa3d7693ba8013 INFO: 172.20.0.1:53458 - "POST /node/69f2c696e2fa3d7693ba8013 HTTP/1.0" 202 Accepted 2026-05-02 03:03:59,166 - lava_callback - INFO - Starting async job submit for node 69f2c696e2fa3d7693ba8013 2026-05-02 03:03:59,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:59,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c696e2fa3d7693ba8013 HTTP/1.1" 200 None 2026-05-02 03:03:59,190 - lava_callback - DEBUG - Processing node 69f2c696e2fa3d7693ba8013: baseline-arm-clabbe 2026-05-02 03:03:59,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c696e2fa3d7693ba8013 2026-05-02 03:03:59,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:59,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c696e2fa3d7693ba8013 HTTP/1.1" 200 None 2026-05-02 03:03:59,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:59,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:03:59,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:03:59,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c696e2fa3d7693ba8013 HTTP/1.1" 200 None 2026-05-02 03:03:59,244 - lava_callback - ERROR - Error processing callback for node 69f2c696e2fa3d7693ba8013: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33756 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:05:18,203 - lava_callback - INFO - Received callback for node 69f2c6e3e2fa3d7693ba802e INFO: 172.20.0.1:33758 - "POST /node/69f2c6e3e2fa3d7693ba802e HTTP/1.0" 202 Accepted 2026-05-02 03:05:18,209 - lava_callback - INFO - Starting async job submit for node 69f2c6e3e2fa3d7693ba802e 2026-05-02 03:05:18,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:05:18,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c6e3e2fa3d7693ba802e HTTP/1.1" 200 None 2026-05-02 03:05:18,223 - lava_callback - DEBUG - Processing node 69f2c6e3e2fa3d7693ba802e: baseline-arm64-baylibre 2026-05-02 03:05:18,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c6e3e2fa3d7693ba802e 2026-05-02 03:05:18,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:05:18,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c6e3e2fa3d7693ba802e HTTP/1.1" 200 None 2026-05-02 03:05:18,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:05:18,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:05:18,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:05:18,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c6e3e2fa3d7693ba802e HTTP/1.1" 200 None 2026-05-02 03:05:18,270 - lava_callback - ERROR - Error processing callback for node 69f2c6e3e2fa3d7693ba802e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52544 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:06:39,308 - lava_callback - INFO - Received callback for node 69f2c72ae2fa3d7693ba8052 INFO: 172.20.0.1:52558 - "POST /node/69f2c72ae2fa3d7693ba8052 HTTP/1.0" 202 Accepted 2026-05-02 03:06:39,313 - lava_callback - INFO - Starting async job submit for node 69f2c72ae2fa3d7693ba8052 2026-05-02 03:06:39,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:39,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c72ae2fa3d7693ba8052 HTTP/1.1" 200 None 2026-05-02 03:06:39,328 - lava_callback - DEBUG - Processing node 69f2c72ae2fa3d7693ba8052: baseline-arm-clabbe 2026-05-02 03:06:39,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c72ae2fa3d7693ba8052 2026-05-02 03:06:39,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:39,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c72ae2fa3d7693ba8052 HTTP/1.1" 200 None 2026-05-02 03:06:39,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:39,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 03:06:39,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:39,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c72ae2fa3d7693ba8052 HTTP/1.1" 200 None 2026-05-02 03:06:39,380 - lava_callback - ERROR - Error processing callback for node 69f2c72ae2fa3d7693ba8052: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59588 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:06:59,277 - lava_callback - INFO - Received callback for node 69f2c74fe2fa3d7693ba806a INFO: 172.20.0.1:59596 - "POST /node/69f2c74fe2fa3d7693ba806a HTTP/1.0" 202 Accepted 2026-05-02 03:06:59,283 - lava_callback - INFO - Starting async job submit for node 69f2c74fe2fa3d7693ba806a 2026-05-02 03:06:59,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:59,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c74fe2fa3d7693ba806a HTTP/1.1" 200 None 2026-05-02 03:06:59,298 - lava_callback - DEBUG - Processing node 69f2c74fe2fa3d7693ba806a: baseline-arm-clabbe 2026-05-02 03:06:59,298 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c74fe2fa3d7693ba806a 2026-05-02 03:06:59,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:59,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c74fe2fa3d7693ba806a HTTP/1.1" 200 None 2026-05-02 03:06:59,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:59,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 03:06:59,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:06:59,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c74fe2fa3d7693ba806a HTTP/1.1" 200 None 2026-05-02 03:06:59,344 - lava_callback - ERROR - Error processing callback for node 69f2c74fe2fa3d7693ba806a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40464 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:07:31,490 - lava_callback - INFO - Received callback for node 69f56a6ded4ed40b02241802 INFO: 172.20.0.1:33694 - "POST /node/69f56a6ded4ed40b02241802 HTTP/1.0" 202 Accepted 2026-05-02 03:07:31,495 - lava_callback - INFO - Starting async job submit for node 69f56a6ded4ed40b02241802 2026-05-02 03:07:31,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:31,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56a6ded4ed40b02241802 HTTP/1.1" 200 None 2026-05-02 03:07:31,511 - lava_callback - DEBUG - Processing node 69f56a6ded4ed40b02241802: baseline-nfs-arm64 2026-05-02 03:07:31,513 - lava_callback - WARNING - Failed to upload log 2026-05-02 03:07:31,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:07:31,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:07:31,607 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f56a6ded4ed40b02241802/lava-logs.yaml 2026-05-02 03:07:31,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:07:31,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:07:31,691 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f56a6ded4ed40b02241802/lava_callback.json.gz 2026-05-02 03:07:31,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:31,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56a6ded4ed40b02241802 HTTP/1.1" 200 None 2026-05-02 03:07:31,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:31,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 03:07:31,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:31,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f56a6ded4ed40b02241802 HTTP/1.1" 200 None 2026-05-02 03:07:31,750 - lava_callback - INFO - Completed processing callback for node 69f56a6ded4ed40b02241802 INFO: 172.20.0.1:33706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38632 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:07:53,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:53,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59738 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:07:58,310 - lava_callback - INFO - Received callback for node 69f2c77be2fa3d7693ba8077 INFO: 172.20.0.1:59750 - "POST /node/69f2c77be2fa3d7693ba8077 HTTP/1.0" 202 Accepted 2026-05-02 03:07:58,316 - lava_callback - INFO - Starting async job submit for node 69f2c77be2fa3d7693ba8077 2026-05-02 03:07:58,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:58,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c77be2fa3d7693ba8077 HTTP/1.1" 200 None 2026-05-02 03:07:58,330 - lava_callback - DEBUG - Processing node 69f2c77be2fa3d7693ba8077: baseline-arm64-baylibre 2026-05-02 03:07:58,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c77be2fa3d7693ba8077 2026-05-02 03:07:58,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:58,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c77be2fa3d7693ba8077 HTTP/1.1" 200 None 2026-05-02 03:07:58,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:58,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:07:58,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:07:58,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c77be2fa3d7693ba8077 HTTP/1.1" 200 None 2026-05-02 03:07:58,390 - lava_callback - ERROR - Error processing callback for node 69f2c77be2fa3d7693ba8077: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32820 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:08:58,292 - lava_callback - INFO - Received callback for node 69f2c7c4e2fa3d7693ba809b INFO: 172.20.0.1:32830 - "POST /node/69f2c7c4e2fa3d7693ba809b HTTP/1.0" 202 Accepted 2026-05-02 03:08:58,297 - lava_callback - INFO - Starting async job submit for node 69f2c7c4e2fa3d7693ba809b 2026-05-02 03:08:58,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:08:58,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c7c4e2fa3d7693ba809b HTTP/1.1" 200 None 2026-05-02 03:08:58,323 - lava_callback - DEBUG - Processing node 69f2c7c4e2fa3d7693ba809b: baseline-arm64-baylibre 2026-05-02 03:08:58,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c7c4e2fa3d7693ba809b 2026-05-02 03:08:58,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:08:58,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c7c4e2fa3d7693ba809b HTTP/1.1" 200 None 2026-05-02 03:08:58,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:08:58,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:08:58,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:08:58,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c7c4e2fa3d7693ba809b HTTP/1.1" 200 None 2026-05-02 03:08:58,372 - lava_callback - ERROR - Error processing callback for node 69f2c7c4e2fa3d7693ba809b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:12:58,479 - lava_callback - INFO - Received callback for node 69f2c8a5e2fa3d7693ba8104 INFO: 172.20.0.1:56036 - "POST /node/69f2c8a5e2fa3d7693ba8104 HTTP/1.0" 202 Accepted 2026-05-02 03:12:58,485 - lava_callback - INFO - Starting async job submit for node 69f2c8a5e2fa3d7693ba8104 2026-05-02 03:12:58,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:12:58,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c8a5e2fa3d7693ba8104 HTTP/1.1" 200 None 2026-05-02 03:12:58,501 - lava_callback - DEBUG - Processing node 69f2c8a5e2fa3d7693ba8104: baseline-arm64-baylibre 2026-05-02 03:12:58,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c8a5e2fa3d7693ba8104 2026-05-02 03:12:58,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:12:58,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c8a5e2fa3d7693ba8104 HTTP/1.1" 200 None 2026-05-02 03:12:58,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:12:58,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:12:58,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:12:58,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c8a5e2fa3d7693ba8104 HTTP/1.1" 200 None 2026-05-02 03:12:58,564 - lava_callback - ERROR - Error processing callback for node 69f2c8a5e2fa3d7693ba8104: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46720 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:14:38,542 - lava_callback - INFO - Received callback for node 69f2c91ae2fa3d7693ba8139 INFO: 172.20.0.1:46734 - "POST /node/69f2c91ae2fa3d7693ba8139 HTTP/1.0" 202 Accepted 2026-05-02 03:14:38,548 - lava_callback - INFO - Starting async job submit for node 69f2c91ae2fa3d7693ba8139 2026-05-02 03:14:38,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:14:38,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c91ae2fa3d7693ba8139 HTTP/1.1" 200 None 2026-05-02 03:14:38,563 - lava_callback - DEBUG - Processing node 69f2c91ae2fa3d7693ba8139: baseline-arm64-baylibre 2026-05-02 03:14:38,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c91ae2fa3d7693ba8139 2026-05-02 03:14:38,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:14:38,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c91ae2fa3d7693ba8139 HTTP/1.1" 200 None 2026-05-02 03:14:38,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:14:38,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:14:38,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:14:38,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c91ae2fa3d7693ba8139 HTTP/1.1" 200 None 2026-05-02 03:14:38,616 - lava_callback - ERROR - Error processing callback for node 69f2c91ae2fa3d7693ba8139: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49840 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:15:58,650 - lava_callback - INFO - Received callback for node 69f2c966e2fa3d7693ba815c INFO: 172.20.0.1:49846 - "POST /node/69f2c966e2fa3d7693ba815c HTTP/1.0" 202 Accepted 2026-05-02 03:15:58,660 - lava_callback - INFO - Starting async job submit for node 69f2c966e2fa3d7693ba815c 2026-05-02 03:15:58,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:15:58,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c966e2fa3d7693ba815c HTTP/1.1" 200 None 2026-05-02 03:15:58,676 - lava_callback - DEBUG - Processing node 69f2c966e2fa3d7693ba815c: baseline-arm64-baylibre 2026-05-02 03:15:58,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c966e2fa3d7693ba815c 2026-05-02 03:15:58,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:15:58,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c966e2fa3d7693ba815c HTTP/1.1" 200 None 2026-05-02 03:15:58,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:15:58,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:15:58,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:15:58,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c966e2fa3d7693ba815c HTTP/1.1" 200 None 2026-05-02 03:15:58,748 - lava_callback - ERROR - Error processing callback for node 69f2c966e2fa3d7693ba815c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47910 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:17:19,743 - lava_callback - INFO - Received callback for node 69f2c9b8e2fa3d7693ba817f INFO: 172.20.0.1:60832 - "POST /node/69f2c9b8e2fa3d7693ba817f HTTP/1.0" 202 Accepted 2026-05-02 03:17:19,749 - lava_callback - INFO - Starting async job submit for node 69f2c9b8e2fa3d7693ba817f 2026-05-02 03:17:19,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:17:19,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c9b8e2fa3d7693ba817f HTTP/1.1" 200 None 2026-05-02 03:17:19,766 - lava_callback - DEBUG - Processing node 69f2c9b8e2fa3d7693ba817f: baseline-arm64-clabbe 2026-05-02 03:17:19,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2c9b8e2fa3d7693ba817f 2026-05-02 03:17:19,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:17:19,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2c9b8e2fa3d7693ba817f HTTP/1.1" 200 None 2026-05-02 03:17:19,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:17:19,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:17:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:17:19,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2c9b8e2fa3d7693ba817f HTTP/1.1" 200 None 2026-05-02 03:17:19,822 - lava_callback - ERROR - Error processing callback for node 69f2c9b8e2fa3d7693ba817f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36918 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:19:37,229 - lava_callback - INFO - Received callback for node 69f56d0eed4ed40b02241929 INFO: 172.20.0.1:36926 - "POST /node/69f56d0eed4ed40b02241929 HTTP/1.0" 202 Accepted 2026-05-02 03:19:37,236 - lava_callback - INFO - Starting async job submit for node 69f56d0eed4ed40b02241929 2026-05-02 03:19:37,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:37,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56d0eed4ed40b02241929 HTTP/1.1" 200 None 2026-05-02 03:19:37,287 - lava_callback - DEBUG - Processing node 69f56d0eed4ed40b02241929: baseline-arm64 2026-05-02 03:19:37,461 - lava_callback - WARNING - Failed to upload log 2026-05-02 03:19:37,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:19:37,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:19:37,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f56d0eed4ed40b02241929/lava-logs.yaml 2026-05-02 03:19:37,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:19:37,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:19:37,773 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f56d0eed4ed40b02241929/lava_callback.json.gz 2026-05-02 03:19:37,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:37,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56d0eed4ed40b02241929 HTTP/1.1" 200 None 2026-05-02 03:19:37,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:37,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 03:19:37,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:37,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f56d0eed4ed40b02241929 HTTP/1.1" 200 None 2026-05-02 03:19:37,917 - lava_callback - INFO - Completed processing callback for node 69f56d0eed4ed40b02241929 INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:19:53,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:53,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56386 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:19:59,894 - lava_callback - INFO - Received callback for node 69f2ca4ee2fa3d7693ba81c9 INFO: 172.20.0.1:55316 - "POST /node/69f2ca4ee2fa3d7693ba81c9 HTTP/1.0" 202 Accepted 2026-05-02 03:19:59,899 - lava_callback - INFO - Starting async job submit for node 69f2ca4ee2fa3d7693ba81c9 2026-05-02 03:19:59,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:59,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ca4ee2fa3d7693ba81c9 HTTP/1.1" 200 None 2026-05-02 03:19:59,918 - lava_callback - DEBUG - Processing node 69f2ca4ee2fa3d7693ba81c9: baseline-arm64-clabbe 2026-05-02 03:19:59,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ca4ee2fa3d7693ba81c9 2026-05-02 03:19:59,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:59,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ca4ee2fa3d7693ba81c9 HTTP/1.1" 200 None 2026-05-02 03:19:59,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:59,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:19:59,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:19:59,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ca4ee2fa3d7693ba81c9 HTTP/1.1" 200 None 2026-05-02 03:19:59,964 - lava_callback - ERROR - Error processing callback for node 69f2ca4ee2fa3d7693ba81c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:21:19,917 - lava_callback - INFO - Received callback for node 69f2ca9ae2fa3d7693ba81f5 INFO: 172.20.0.1:37514 - "POST /node/69f2ca9ae2fa3d7693ba81f5 HTTP/1.0" 202 Accepted 2026-05-02 03:21:19,926 - lava_callback - INFO - Starting async job submit for node 69f2ca9ae2fa3d7693ba81f5 2026-05-02 03:21:19,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:21:19,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ca9ae2fa3d7693ba81f5 HTTP/1.1" 200 None 2026-05-02 03:21:19,944 - lava_callback - DEBUG - Processing node 69f2ca9ae2fa3d7693ba81f5: baseline-arm64-clabbe 2026-05-02 03:21:19,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ca9ae2fa3d7693ba81f5 2026-05-02 03:21:19,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:21:19,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ca9ae2fa3d7693ba81f5 HTTP/1.1" 200 None 2026-05-02 03:21:19,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:21:19,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:21:19,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:21:19,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ca9ae2fa3d7693ba81f5 HTTP/1.1" 200 None 2026-05-02 03:21:20,010 - lava_callback - ERROR - Error processing callback for node 69f2ca9ae2fa3d7693ba81f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54384 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:23:11,680 - lava_callback - INFO - Received callback for node 69f56df3ed4ed40b0224198f INFO: 172.20.0.1:51912 - "POST /node/69f56df3ed4ed40b0224198f HTTP/1.0" 202 Accepted 2026-05-02 03:23:11,686 - lava_callback - INFO - Starting async job submit for node 69f56df3ed4ed40b0224198f 2026-05-02 03:23:11,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:11,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56df3ed4ed40b0224198f HTTP/1.1" 200 None 2026-05-02 03:23:11,729 - lava_callback - DEBUG - Processing node 69f56df3ed4ed40b0224198f: baseline-arm64 2026-05-02 03:23:11,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:23:11,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:23:11,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f56df3ed4ed40b0224198f/log.txt.gz 2026-05-02 03:23:12,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:23:12,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:23:12,177 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f56df3ed4ed40b0224198f/lava-logs.yaml 2026-05-02 03:23:12,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:23:12,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:23:12,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f56df3ed4ed40b0224198f/lava_callback.json.gz 2026-05-02 03:23:12,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:12,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f56df3ed4ed40b0224198f HTTP/1.1" 200 None 2026-05-02 03:23:12,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:12,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 03:23:12,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:12,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f56df3ed4ed40b0224198f HTTP/1.1" 200 None 2026-05-02 03:23:12,416 - lava_callback - INFO - Completed processing callback for node 69f56df3ed4ed40b0224198f INFO: 172.20.0.1:51924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51934 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:23:23,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:23,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34288 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:23:40,333 - lava_callback - INFO - Received callback for node 69f2cb32e2fa3d7693ba8233 INFO: 172.20.0.1:46812 - "POST /node/69f2cb32e2fa3d7693ba8233 HTTP/1.0" 202 Accepted 2026-05-02 03:23:40,339 - lava_callback - INFO - Starting async job submit for node 69f2cb32e2fa3d7693ba8233 2026-05-02 03:23:40,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:40,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cb32e2fa3d7693ba8233 HTTP/1.1" 200 None 2026-05-02 03:23:40,360 - lava_callback - DEBUG - Processing node 69f2cb32e2fa3d7693ba8233: baseline-arm64-clabbe 2026-05-02 03:23:40,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cb32e2fa3d7693ba8233 2026-05-02 03:23:40,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:40,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cb32e2fa3d7693ba8233 HTTP/1.1" 200 None 2026-05-02 03:23:40,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:40,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:23:40,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:23:40,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cb32e2fa3d7693ba8233 HTTP/1.1" 200 None 2026-05-02 03:23:40,413 - lava_callback - ERROR - Error processing callback for node 69f2cb32e2fa3d7693ba8233: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60204 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:25:00,159 - lava_callback - INFO - Received callback for node 69f2cb7ee2fa3d7693ba8251 INFO: 172.20.0.1:53318 - "POST /node/69f2cb7ee2fa3d7693ba8251 HTTP/1.0" 202 Accepted 2026-05-02 03:25:00,165 - lava_callback - INFO - Starting async job submit for node 69f2cb7ee2fa3d7693ba8251 2026-05-02 03:25:00,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:25:00,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cb7ee2fa3d7693ba8251 HTTP/1.1" 200 None 2026-05-02 03:25:00,180 - lava_callback - DEBUG - Processing node 69f2cb7ee2fa3d7693ba8251: baseline-arm64-clabbe 2026-05-02 03:25:00,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cb7ee2fa3d7693ba8251 2026-05-02 03:25:00,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:25:00,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cb7ee2fa3d7693ba8251 HTTP/1.1" 200 None 2026-05-02 03:25:00,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:25:00,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:25:00,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:25:00,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cb7ee2fa3d7693ba8251 HTTP/1.1" 200 None 2026-05-02 03:25:00,228 - lava_callback - ERROR - Error processing callback for node 69f2cb7ee2fa3d7693ba8251: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41570 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:26:20,238 - lava_callback - INFO - Received callback for node 69f2cbcbe2fa3d7693ba8273 INFO: 172.20.0.1:52710 - "POST /node/69f2cbcbe2fa3d7693ba8273 HTTP/1.0" 202 Accepted 2026-05-02 03:26:20,243 - lava_callback - INFO - Starting async job submit for node 69f2cbcbe2fa3d7693ba8273 2026-05-02 03:26:20,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:26:20,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cbcbe2fa3d7693ba8273 HTTP/1.1" 200 None 2026-05-02 03:26:20,258 - lava_callback - DEBUG - Processing node 69f2cbcbe2fa3d7693ba8273: baseline-arm64-clabbe 2026-05-02 03:26:20,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cbcbe2fa3d7693ba8273 2026-05-02 03:26:20,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:26:20,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cbcbe2fa3d7693ba8273 HTTP/1.1" 200 None 2026-05-02 03:26:20,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:26:20,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:26:20,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:26:20,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cbcbe2fa3d7693ba8273 HTTP/1.1" 200 None 2026-05-02 03:26:20,308 - lava_callback - ERROR - Error processing callback for node 69f2cbcbe2fa3d7693ba8273: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48412 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:27:40,282 - lava_callback - INFO - Received callback for node 69f2cc17e2fa3d7693ba8290 INFO: 172.20.0.1:52426 - "POST /node/69f2cc17e2fa3d7693ba8290 HTTP/1.0" 202 Accepted 2026-05-02 03:27:40,288 - lava_callback - INFO - Starting async job submit for node 69f2cc17e2fa3d7693ba8290 2026-05-02 03:27:40,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:27:40,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc17e2fa3d7693ba8290 HTTP/1.1" 200 None 2026-05-02 03:27:40,303 - lava_callback - DEBUG - Processing node 69f2cc17e2fa3d7693ba8290: baseline-arm64-clabbe 2026-05-02 03:27:40,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cc17e2fa3d7693ba8290 2026-05-02 03:27:40,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:27:40,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc17e2fa3d7693ba8290 HTTP/1.1" 200 None 2026-05-02 03:27:40,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:27:40,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 03:27:40,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:27:40,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cc17e2fa3d7693ba8290 HTTP/1.1" 200 None 2026-05-02 03:27:40,356 - lava_callback - ERROR - Error processing callback for node 69f2cc17e2fa3d7693ba8290: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33208 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:28:19,333 - lava_callback - INFO - Received callback for node 69f2cc50e2fa3d7693ba82af INFO: 172.20.0.1:33214 - "POST /node/69f2cc50e2fa3d7693ba82af HTTP/1.0" 202 Accepted 2026-05-02 03:28:19,339 - lava_callback - INFO - Starting async job submit for node 69f2cc50e2fa3d7693ba82af 2026-05-02 03:28:19,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:19,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc50e2fa3d7693ba82af HTTP/1.1" 200 None 2026-05-02 03:28:19,358 - lava_callback - DEBUG - Processing node 69f2cc50e2fa3d7693ba82af: baseline-arm-baylibre 2026-05-02 03:28:19,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cc50e2fa3d7693ba82af 2026-05-02 03:28:19,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:19,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc50e2fa3d7693ba82af HTTP/1.1" 200 None 2026-05-02 03:28:19,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:19,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:28:19,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:19,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cc50e2fa3d7693ba82af HTTP/1.1" 200 None 2026-05-02 03:28:19,405 - lava_callback - ERROR - Error processing callback for node 69f2cc50e2fa3d7693ba82af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58660 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:28:59,236 - lava_callback - INFO - Received callback for node 69f2cc76e2fa3d7693ba82bc INFO: 172.20.0.1:58676 - "POST /node/69f2cc76e2fa3d7693ba82bc HTTP/1.0" 202 Accepted 2026-05-02 03:28:59,242 - lava_callback - INFO - Starting async job submit for node 69f2cc76e2fa3d7693ba82bc 2026-05-02 03:28:59,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:59,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc76e2fa3d7693ba82bc HTTP/1.1" 200 None 2026-05-02 03:28:59,343 - lava_callback - DEBUG - Processing node 69f2cc76e2fa3d7693ba82bc: baseline-arm-baylibre 2026-05-02 03:28:59,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cc76e2fa3d7693ba82bc 2026-05-02 03:28:59,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:59,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc76e2fa3d7693ba82bc HTTP/1.1" 200 None 2026-05-02 03:28:59,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:59,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:28:59,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:28:59,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cc76e2fa3d7693ba82bc HTTP/1.1" 200 None 2026-05-02 03:28:59,395 - lava_callback - ERROR - Error processing callback for node 69f2cc76e2fa3d7693ba82bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:29:39,322 - lava_callback - INFO - Received callback for node 69f2cc9ce2fa3d7693ba82ca INFO: 172.20.0.1:55880 - "POST /node/69f2cc9ce2fa3d7693ba82ca HTTP/1.0" 202 Accepted 2026-05-02 03:29:39,328 - lava_callback - INFO - Starting async job submit for node 69f2cc9ce2fa3d7693ba82ca 2026-05-02 03:29:39,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:29:39,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc9ce2fa3d7693ba82ca HTTP/1.1" 200 None 2026-05-02 03:29:39,343 - lava_callback - DEBUG - Processing node 69f2cc9ce2fa3d7693ba82ca: baseline-arm-baylibre 2026-05-02 03:29:39,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cc9ce2fa3d7693ba82ca 2026-05-02 03:29:39,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:29:39,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cc9ce2fa3d7693ba82ca HTTP/1.1" 200 None 2026-05-02 03:29:39,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:29:39,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:29:39,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:29:39,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cc9ce2fa3d7693ba82ca HTTP/1.1" 200 None 2026-05-02 03:29:39,401 - lava_callback - ERROR - Error processing callback for node 69f2cc9ce2fa3d7693ba82ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53150 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:30:19,284 - lava_callback - INFO - Received callback for node 69f2ccc3e2fa3d7693ba82dc INFO: 172.20.0.1:53158 - "POST /node/69f2ccc3e2fa3d7693ba82dc HTTP/1.0" 202 Accepted 2026-05-02 03:30:19,289 - lava_callback - INFO - Starting async job submit for node 69f2ccc3e2fa3d7693ba82dc 2026-05-02 03:30:19,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:19,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ccc3e2fa3d7693ba82dc HTTP/1.1" 200 None 2026-05-02 03:30:19,304 - lava_callback - DEBUG - Processing node 69f2ccc3e2fa3d7693ba82dc: baseline-arm-baylibre 2026-05-02 03:30:19,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ccc3e2fa3d7693ba82dc 2026-05-02 03:30:19,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:19,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ccc3e2fa3d7693ba82dc HTTP/1.1" 200 None 2026-05-02 03:30:19,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:30:19,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:19,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ccc3e2fa3d7693ba82dc HTTP/1.1" 200 None 2026-05-02 03:30:19,358 - lava_callback - ERROR - Error processing callback for node 69f2ccc3e2fa3d7693ba82dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40076 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:30:59,414 - lava_callback - INFO - Received callback for node 69f2ccece2fa3d7693ba82e9 INFO: 172.20.0.1:40082 - "POST /node/69f2ccece2fa3d7693ba82e9 HTTP/1.0" 202 Accepted 2026-05-02 03:30:59,420 - lava_callback - INFO - Starting async job submit for node 69f2ccece2fa3d7693ba82e9 2026-05-02 03:30:59,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:59,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ccece2fa3d7693ba82e9 HTTP/1.1" 200 None 2026-05-02 03:30:59,439 - lava_callback - DEBUG - Processing node 69f2ccece2fa3d7693ba82e9: baseline-arm-baylibre 2026-05-02 03:30:59,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ccece2fa3d7693ba82e9 2026-05-02 03:30:59,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:59,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ccece2fa3d7693ba82e9 HTTP/1.1" 200 None 2026-05-02 03:30:59,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:59,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:30:59,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:30:59,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ccece2fa3d7693ba82e9 HTTP/1.1" 200 None 2026-05-02 03:30:59,487 - lava_callback - ERROR - Error processing callback for node 69f2ccece2fa3d7693ba82e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34174 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:31:39,388 - lava_callback - INFO - Received callback for node 69f2cd11e2fa3d7693ba82f5 INFO: 172.20.0.1:34190 - "POST /node/69f2cd11e2fa3d7693ba82f5 HTTP/1.0" 202 Accepted 2026-05-02 03:31:39,394 - lava_callback - INFO - Starting async job submit for node 69f2cd11e2fa3d7693ba82f5 2026-05-02 03:31:39,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:31:39,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd11e2fa3d7693ba82f5 HTTP/1.1" 200 None 2026-05-02 03:31:39,415 - lava_callback - DEBUG - Processing node 69f2cd11e2fa3d7693ba82f5: baseline-arm-baylibre 2026-05-02 03:31:39,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cd11e2fa3d7693ba82f5 2026-05-02 03:31:39,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:31:39,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd11e2fa3d7693ba82f5 HTTP/1.1" 200 None 2026-05-02 03:31:39,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:31:39,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:31:39,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:31:39,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cd11e2fa3d7693ba82f5 HTTP/1.1" 200 None 2026-05-02 03:31:39,475 - lava_callback - ERROR - Error processing callback for node 69f2cd11e2fa3d7693ba82f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49330 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:32:20,486 - lava_callback - INFO - Received callback for node 69f2cd37e2fa3d7693ba8302 INFO: 172.20.0.1:38364 - "POST /node/69f2cd37e2fa3d7693ba8302 HTTP/1.0" 202 Accepted 2026-05-02 03:32:20,491 - lava_callback - INFO - Starting async job submit for node 69f2cd37e2fa3d7693ba8302 2026-05-02 03:32:20,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:20,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd37e2fa3d7693ba8302 HTTP/1.1" 200 None 2026-05-02 03:32:20,506 - lava_callback - DEBUG - Processing node 69f2cd37e2fa3d7693ba8302: baseline-arm-clabbe 2026-05-02 03:32:20,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cd37e2fa3d7693ba8302 2026-05-02 03:32:20,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:20,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd37e2fa3d7693ba8302 HTTP/1.1" 200 None 2026-05-02 03:32:20,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:20,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:32:20,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:20,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cd37e2fa3d7693ba8302 HTTP/1.1" 200 None 2026-05-02 03:32:20,556 - lava_callback - ERROR - Error processing callback for node 69f2cd37e2fa3d7693ba8302: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:32:59,522 - lava_callback - INFO - Received callback for node 69f2cd58e2fa3d7693ba830f INFO: 172.20.0.1:49338 - "POST /node/69f2cd58e2fa3d7693ba830f HTTP/1.0" 202 Accepted 2026-05-02 03:32:59,527 - lava_callback - INFO - Starting async job submit for node 69f2cd58e2fa3d7693ba830f 2026-05-02 03:32:59,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:59,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd58e2fa3d7693ba830f HTTP/1.1" 200 None 2026-05-02 03:32:59,543 - lava_callback - DEBUG - Processing node 69f2cd58e2fa3d7693ba830f: baseline-arm-baylibre 2026-05-02 03:32:59,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cd58e2fa3d7693ba830f 2026-05-02 03:32:59,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:59,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd58e2fa3d7693ba830f HTTP/1.1" 200 None 2026-05-02 03:32:59,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:59,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:32:59,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:32:59,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cd58e2fa3d7693ba830f HTTP/1.1" 200 None 2026-05-02 03:32:59,592 - lava_callback - ERROR - Error processing callback for node 69f2cd58e2fa3d7693ba830f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41234 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:33:40,603 - lava_callback - INFO - Received callback for node 69f2cd81e2fa3d7693ba831b INFO: 172.20.0.1:49248 - "POST /node/69f2cd81e2fa3d7693ba831b HTTP/1.0" 202 Accepted 2026-05-02 03:33:40,613 - lava_callback - INFO - Starting async job submit for node 69f2cd81e2fa3d7693ba831b 2026-05-02 03:33:40,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:33:40,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd81e2fa3d7693ba831b HTTP/1.1" 200 None 2026-05-02 03:33:40,630 - lava_callback - DEBUG - Processing node 69f2cd81e2fa3d7693ba831b: baseline-arm-clabbe 2026-05-02 03:33:40,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cd81e2fa3d7693ba831b 2026-05-02 03:33:40,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:33:40,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cd81e2fa3d7693ba831b HTTP/1.1" 200 None 2026-05-02 03:33:40,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:33:40,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:33:40,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:33:40,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cd81e2fa3d7693ba831b HTTP/1.1" 200 None 2026-05-02 03:33:40,676 - lava_callback - ERROR - Error processing callback for node 69f2cd81e2fa3d7693ba831b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58624 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:34:00,532 - lava_callback - INFO - Received callback for node 69f2cda5e2fa3d7693ba832b INFO: 172.20.0.1:55754 - "POST /node/69f2cda5e2fa3d7693ba832b HTTP/1.0" 202 Accepted 2026-05-02 03:34:00,541 - lava_callback - INFO - Starting async job submit for node 69f2cda5e2fa3d7693ba832b 2026-05-02 03:34:00,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:00,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cda5e2fa3d7693ba832b HTTP/1.1" 200 None 2026-05-02 03:34:00,557 - lava_callback - DEBUG - Processing node 69f2cda5e2fa3d7693ba832b: baseline-arm-clabbe 2026-05-02 03:34:00,557 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cda5e2fa3d7693ba832b 2026-05-02 03:34:00,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:00,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cda5e2fa3d7693ba832b HTTP/1.1" 200 None 2026-05-02 03:34:00,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:00,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:34:00,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:00,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cda5e2fa3d7693ba832b HTTP/1.1" 200 None 2026-05-02 03:34:00,610 - lava_callback - ERROR - Error processing callback for node 69f2cda5e2fa3d7693ba832b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52992 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:34:39,583 - lava_callback - INFO - Received callback for node 69f2cdc6e2fa3d7693ba8339 INFO: 172.20.0.1:53004 - "POST /node/69f2cdc6e2fa3d7693ba8339 HTTP/1.0" 202 Accepted 2026-05-02 03:34:39,589 - lava_callback - INFO - Starting async job submit for node 69f2cdc6e2fa3d7693ba8339 2026-05-02 03:34:39,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:39,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cdc6e2fa3d7693ba8339 HTTP/1.1" 200 None 2026-05-02 03:34:39,605 - lava_callback - DEBUG - Processing node 69f2cdc6e2fa3d7693ba8339: baseline-arm-baylibre 2026-05-02 03:34:39,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cdc6e2fa3d7693ba8339 2026-05-02 03:34:39,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:39,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cdc6e2fa3d7693ba8339 HTTP/1.1" 200 None 2026-05-02 03:34:39,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:39,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:34:39,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:34:39,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cdc6e2fa3d7693ba8339 HTTP/1.1" 200 None 2026-05-02 03:34:39,658 - lava_callback - ERROR - Error processing callback for node 69f2cdc6e2fa3d7693ba8339: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45570 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:35:19,578 - lava_callback - INFO - Received callback for node 69f2cdeae2fa3d7693ba834a INFO: 172.20.0.1:45578 - "POST /node/69f2cdeae2fa3d7693ba834a HTTP/1.0" 202 Accepted 2026-05-02 03:35:19,584 - lava_callback - INFO - Starting async job submit for node 69f2cdeae2fa3d7693ba834a 2026-05-02 03:35:19,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:35:19,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cdeae2fa3d7693ba834a HTTP/1.1" 200 None 2026-05-02 03:35:19,605 - lava_callback - DEBUG - Processing node 69f2cdeae2fa3d7693ba834a: baseline-arm-baylibre 2026-05-02 03:35:19,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cdeae2fa3d7693ba834a 2026-05-02 03:35:19,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:35:19,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cdeae2fa3d7693ba834a HTTP/1.1" 200 None 2026-05-02 03:35:19,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:35:19,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:35:19,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:35:19,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cdeae2fa3d7693ba834a HTTP/1.1" 200 None 2026-05-02 03:35:19,652 - lava_callback - ERROR - Error processing callback for node 69f2cdeae2fa3d7693ba834a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33152 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:36:00,651 - lava_callback - INFO - Received callback for node 69f2ce12e2fa3d7693ba8358 INFO: 172.20.0.1:55428 - "POST /node/69f2ce12e2fa3d7693ba8358 HTTP/1.0" 202 Accepted 2026-05-02 03:36:00,656 - lava_callback - INFO - Starting async job submit for node 69f2ce12e2fa3d7693ba8358 2026-05-02 03:36:00,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:00,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce12e2fa3d7693ba8358 HTTP/1.1" 200 None 2026-05-02 03:36:00,678 - lava_callback - DEBUG - Processing node 69f2ce12e2fa3d7693ba8358: baseline-arm-clabbe 2026-05-02 03:36:00,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ce12e2fa3d7693ba8358 2026-05-02 03:36:00,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:00,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce12e2fa3d7693ba8358 HTTP/1.1" 200 None 2026-05-02 03:36:00,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:00,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:36:00,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:00,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ce12e2fa3d7693ba8358 HTTP/1.1" 200 None 2026-05-02 03:36:00,728 - lava_callback - ERROR - Error processing callback for node 69f2ce12e2fa3d7693ba8358: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39892 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:36:39,668 - lava_callback - INFO - Received callback for node 69f2ce34e2fa3d7693ba8363 INFO: 172.20.0.1:39898 - "POST /node/69f2ce34e2fa3d7693ba8363 HTTP/1.0" 202 Accepted 2026-05-02 03:36:39,674 - lava_callback - INFO - Starting async job submit for node 69f2ce34e2fa3d7693ba8363 2026-05-02 03:36:39,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:39,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce34e2fa3d7693ba8363 HTTP/1.1" 200 None 2026-05-02 03:36:39,688 - lava_callback - DEBUG - Processing node 69f2ce34e2fa3d7693ba8363: baseline-arm-baylibre 2026-05-02 03:36:39,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ce34e2fa3d7693ba8363 2026-05-02 03:36:39,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:39,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce34e2fa3d7693ba8363 HTTP/1.1" 200 None 2026-05-02 03:36:39,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:39,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:36:39,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:36:39,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ce34e2fa3d7693ba8363 HTTP/1.1" 200 None 2026-05-02 03:36:39,771 - lava_callback - ERROR - Error processing callback for node 69f2ce34e2fa3d7693ba8363: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55094 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:37:20,731 - lava_callback - INFO - Received callback for node 69f2ce5ce2fa3d7693ba8373 INFO: 172.20.0.1:54578 - "POST /node/69f2ce5ce2fa3d7693ba8373 HTTP/1.0" 202 Accepted 2026-05-02 03:37:20,739 - lava_callback - INFO - Starting async job submit for node 69f2ce5ce2fa3d7693ba8373 2026-05-02 03:37:20,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:20,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce5ce2fa3d7693ba8373 HTTP/1.1" 200 None 2026-05-02 03:37:20,759 - lava_callback - DEBUG - Processing node 69f2ce5ce2fa3d7693ba8373: baseline-arm-clabbe 2026-05-02 03:37:20,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ce5ce2fa3d7693ba8373 2026-05-02 03:37:20,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:20,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce5ce2fa3d7693ba8373 HTTP/1.1" 200 None 2026-05-02 03:37:20,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:20,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:37:20,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:20,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ce5ce2fa3d7693ba8373 HTTP/1.1" 200 None 2026-05-02 03:37:20,810 - lava_callback - ERROR - Error processing callback for node 69f2ce5ce2fa3d7693ba8373: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:37:40,667 - lava_callback - INFO - Received callback for node 69f2ce81e2fa3d7693ba837c INFO: 172.20.0.1:50626 - "POST /node/69f2ce81e2fa3d7693ba837c HTTP/1.0" 202 Accepted 2026-05-02 03:37:40,673 - lava_callback - INFO - Starting async job submit for node 69f2ce81e2fa3d7693ba837c 2026-05-02 03:37:40,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:40,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce81e2fa3d7693ba837c HTTP/1.1" 200 None 2026-05-02 03:37:40,687 - lava_callback - DEBUG - Processing node 69f2ce81e2fa3d7693ba837c: baseline-arm-clabbe 2026-05-02 03:37:40,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ce81e2fa3d7693ba837c 2026-05-02 03:37:40,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:40,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ce81e2fa3d7693ba837c HTTP/1.1" 200 None 2026-05-02 03:37:40,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:40,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:37:40,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:37:40,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ce81e2fa3d7693ba837c HTTP/1.1" 200 None 2026-05-02 03:37:40,750 - lava_callback - ERROR - Error processing callback for node 69f2ce81e2fa3d7693ba837c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55438 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:38:19,769 - lava_callback - INFO - Received callback for node 69f2cea3e2fa3d7693ba8387 INFO: 172.20.0.1:57960 - "POST /node/69f2cea3e2fa3d7693ba8387 HTTP/1.0" 202 Accepted 2026-05-02 03:38:19,775 - lava_callback - INFO - Starting async job submit for node 69f2cea3e2fa3d7693ba8387 2026-05-02 03:38:19,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:19,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cea3e2fa3d7693ba8387 HTTP/1.1" 200 None 2026-05-02 03:38:19,789 - lava_callback - DEBUG - Processing node 69f2cea3e2fa3d7693ba8387: baseline-arm-baylibre 2026-05-02 03:38:19,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cea3e2fa3d7693ba8387 2026-05-02 03:38:19,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:19,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cea3e2fa3d7693ba8387 HTTP/1.1" 200 None 2026-05-02 03:38:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:19,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:38:19,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:19,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cea3e2fa3d7693ba8387 HTTP/1.1" 200 None 2026-05-02 03:38:19,839 - lava_callback - ERROR - Error processing callback for node 69f2cea3e2fa3d7693ba8387: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53918 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:38:59,814 - lava_callback - INFO - Received callback for node 69f2cecde2fa3d7693ba8395 INFO: 172.20.0.1:42748 - "POST /node/69f2cecde2fa3d7693ba8395 HTTP/1.0" 202 Accepted 2026-05-02 03:38:59,819 - lava_callback - INFO - Starting async job submit for node 69f2cecde2fa3d7693ba8395 2026-05-02 03:38:59,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:59,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cecde2fa3d7693ba8395 HTTP/1.1" 200 None 2026-05-02 03:38:59,834 - lava_callback - DEBUG - Processing node 69f2cecde2fa3d7693ba8395: baseline-arm-baylibre 2026-05-02 03:38:59,834 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cecde2fa3d7693ba8395 2026-05-02 03:38:59,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:59,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cecde2fa3d7693ba8395 HTTP/1.1" 200 None 2026-05-02 03:38:59,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:59,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:38:59,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:38:59,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cecde2fa3d7693ba8395 HTTP/1.1" 200 None 2026-05-02 03:38:59,884 - lava_callback - ERROR - Error processing callback for node 69f2cecde2fa3d7693ba8395: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33236 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:39:40,859 - lava_callback - INFO - Received callback for node 69f2cef3e2fa3d7693ba83a2 INFO: 172.20.0.1:44938 - "POST /node/69f2cef3e2fa3d7693ba83a2 HTTP/1.0" 202 Accepted 2026-05-02 03:39:40,864 - lava_callback - INFO - Starting async job submit for node 69f2cef3e2fa3d7693ba83a2 2026-05-02 03:39:40,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:39:40,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cef3e2fa3d7693ba83a2 HTTP/1.1" 200 None 2026-05-02 03:39:40,880 - lava_callback - DEBUG - Processing node 69f2cef3e2fa3d7693ba83a2: baseline-arm-clabbe 2026-05-02 03:39:40,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cef3e2fa3d7693ba83a2 2026-05-02 03:39:40,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:39:40,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cef3e2fa3d7693ba83a2 HTTP/1.1" 200 None 2026-05-02 03:39:40,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:39:40,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:39:40,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:39:40,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cef3e2fa3d7693ba83a2 HTTP/1.1" 200 None 2026-05-02 03:39:40,947 - lava_callback - ERROR - Error processing callback for node 69f2cef3e2fa3d7693ba83a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54536 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:40:20,844 - lava_callback - INFO - Received callback for node 69f2cf17e2fa3d7693ba83ae INFO: 172.20.0.1:55606 - "POST /node/69f2cf17e2fa3d7693ba83ae HTTP/1.0" 202 Accepted 2026-05-02 03:40:20,850 - lava_callback - INFO - Starting async job submit for node 69f2cf17e2fa3d7693ba83ae 2026-05-02 03:40:20,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:40:20,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf17e2fa3d7693ba83ae HTTP/1.1" 200 None 2026-05-02 03:40:20,866 - lava_callback - DEBUG - Processing node 69f2cf17e2fa3d7693ba83ae: baseline-arm-clabbe 2026-05-02 03:40:20,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cf17e2fa3d7693ba83ae 2026-05-02 03:40:20,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:40:20,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf17e2fa3d7693ba83ae HTTP/1.1" 200 None 2026-05-02 03:40:20,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:40:20,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:40:20,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:40:20,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cf17e2fa3d7693ba83ae HTTP/1.1" 200 None 2026-05-02 03:40:20,911 - lava_callback - ERROR - Error processing callback for node 69f2cf17e2fa3d7693ba83ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59568 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:40:59,957 - lava_callback - INFO - Received callback for node 69f2cf37e2fa3d7693ba83bb INFO: 172.20.0.1:60798 - "POST /node/69f2cf37e2fa3d7693ba83bb HTTP/1.0" 202 Accepted 2026-05-02 03:40:59,962 - lava_callback - INFO - Starting async job submit for node 69f2cf37e2fa3d7693ba83bb 2026-05-02 03:40:59,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:40:59,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf37e2fa3d7693ba83bb HTTP/1.1" 200 None 2026-05-02 03:40:59,989 - lava_callback - DEBUG - Processing node 69f2cf37e2fa3d7693ba83bb: baseline-arm-baylibre 2026-05-02 03:40:59,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cf37e2fa3d7693ba83bb 2026-05-02 03:40:59,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:41:00,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf37e2fa3d7693ba83bb HTTP/1.1" 200 None 2026-05-02 03:41:00,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:41:00,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:41:00,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:41:00,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cf37e2fa3d7693ba83bb HTTP/1.1" 200 None 2026-05-02 03:41:00,035 - lava_callback - ERROR - Error processing callback for node 69f2cf37e2fa3d7693ba83bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44540 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:41:40,826 - lava_callback - INFO - Received callback for node 69f2cf5fe2fa3d7693ba83ca INFO: 172.20.0.1:55614 - "POST /node/69f2cf5fe2fa3d7693ba83ca HTTP/1.0" 202 Accepted 2026-05-02 03:41:40,831 - lava_callback - INFO - Starting async job submit for node 69f2cf5fe2fa3d7693ba83ca 2026-05-02 03:41:40,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:41:40,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf5fe2fa3d7693ba83ca HTTP/1.1" 200 None 2026-05-02 03:41:40,846 - lava_callback - DEBUG - Processing node 69f2cf5fe2fa3d7693ba83ca: baseline-arm-clabbe 2026-05-02 03:41:40,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cf5fe2fa3d7693ba83ca 2026-05-02 03:41:40,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:41:40,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf5fe2fa3d7693ba83ca HTTP/1.1" 200 None 2026-05-02 03:41:40,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:41:40,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:41:40,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:41:40,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cf5fe2fa3d7693ba83ca HTTP/1.1" 200 None 2026-05-02 03:41:40,890 - lava_callback - ERROR - Error processing callback for node 69f2cf5fe2fa3d7693ba83ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46442 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:42:00,854 - lava_callback - INFO - Received callback for node 69f2cf84e2fa3d7693ba83da INFO: 172.20.0.1:41554 - "POST /node/69f2cf84e2fa3d7693ba83da HTTP/1.0" 202 Accepted 2026-05-02 03:42:00,860 - lava_callback - INFO - Starting async job submit for node 69f2cf84e2fa3d7693ba83da 2026-05-02 03:42:00,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:00,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf84e2fa3d7693ba83da HTTP/1.1" 200 None 2026-05-02 03:42:00,876 - lava_callback - DEBUG - Processing node 69f2cf84e2fa3d7693ba83da: baseline-arm-clabbe 2026-05-02 03:42:00,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cf84e2fa3d7693ba83da 2026-05-02 03:42:00,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:00,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cf84e2fa3d7693ba83da HTTP/1.1" 200 None 2026-05-02 03:42:00,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:00,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 03:42:00,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:00,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cf84e2fa3d7693ba83da HTTP/1.1" 200 None 2026-05-02 03:42:00,946 - lava_callback - ERROR - Error processing callback for node 69f2cf84e2fa3d7693ba83da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45552 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:42:40,031 - lava_callback - INFO - Received callback for node 69f2cfa4e2fa3d7693ba83e3 INFO: 172.20.0.1:47182 - "POST /node/69f2cfa4e2fa3d7693ba83e3 HTTP/1.0" 202 Accepted 2026-05-02 03:42:40,036 - lava_callback - INFO - Starting async job submit for node 69f2cfa4e2fa3d7693ba83e3 2026-05-02 03:42:40,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:40,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cfa4e2fa3d7693ba83e3 HTTP/1.1" 200 None 2026-05-02 03:42:40,053 - lava_callback - DEBUG - Processing node 69f2cfa4e2fa3d7693ba83e3: baseline-arm-baylibre 2026-05-02 03:42:40,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cfa4e2fa3d7693ba83e3 2026-05-02 03:42:40,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:40,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cfa4e2fa3d7693ba83e3 HTTP/1.1" 200 None 2026-05-02 03:42:40,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:40,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:42:40,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:42:40,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cfa4e2fa3d7693ba83e3 HTTP/1.1" 200 None 2026-05-02 03:42:40,103 - lava_callback - ERROR - Error processing callback for node 69f2cfa4e2fa3d7693ba83e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49702 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:43:20,948 - lava_callback - INFO - Received callback for node 69f2cfcde2fa3d7693ba83f1 INFO: 172.20.0.1:55564 - "POST /node/69f2cfcde2fa3d7693ba83f1 HTTP/1.0" 202 Accepted 2026-05-02 03:43:20,953 - lava_callback - INFO - Starting async job submit for node 69f2cfcde2fa3d7693ba83f1 2026-05-02 03:43:20,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:43:20,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cfcde2fa3d7693ba83f1 HTTP/1.1" 200 None 2026-05-02 03:43:20,976 - lava_callback - DEBUG - Processing node 69f2cfcde2fa3d7693ba83f1: baseline-arm-clabbe 2026-05-02 03:43:20,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cfcde2fa3d7693ba83f1 2026-05-02 03:43:20,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:43:20,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cfcde2fa3d7693ba83f1 HTTP/1.1" 200 None 2026-05-02 03:43:20,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:43:21,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:43:21,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:43:21,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cfcde2fa3d7693ba83f1 HTTP/1.1" 200 None 2026-05-02 03:43:21,023 - lava_callback - ERROR - Error processing callback for node 69f2cfcde2fa3d7693ba83f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33648 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:44:00,943 - lava_callback - INFO - Received callback for node 69f2cff2e2fa3d7693ba83ff INFO: 172.20.0.1:51852 - "POST /node/69f2cff2e2fa3d7693ba83ff HTTP/1.0" 202 Accepted 2026-05-02 03:44:00,950 - lava_callback - INFO - Starting async job submit for node 69f2cff2e2fa3d7693ba83ff 2026-05-02 03:44:00,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:00,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cff2e2fa3d7693ba83ff HTTP/1.1" 200 None 2026-05-02 03:44:00,973 - lava_callback - DEBUG - Processing node 69f2cff2e2fa3d7693ba83ff: baseline-arm-clabbe 2026-05-02 03:44:00,974 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2cff2e2fa3d7693ba83ff 2026-05-02 03:44:00,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:00,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2cff2e2fa3d7693ba83ff HTTP/1.1" 200 None 2026-05-02 03:44:00,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:01,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:44:01,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:01,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2cff2e2fa3d7693ba83ff HTTP/1.1" 200 None 2026-05-02 03:44:01,037 - lava_callback - ERROR - Error processing callback for node 69f2cff2e2fa3d7693ba83ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35786 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:44:40,155 - lava_callback - INFO - Received callback for node 69f2d013e2fa3d7693ba840a INFO: 172.20.0.1:40142 - "POST /node/69f2d013e2fa3d7693ba840a HTTP/1.0" 202 Accepted 2026-05-02 03:44:40,160 - lava_callback - INFO - Starting async job submit for node 69f2d013e2fa3d7693ba840a 2026-05-02 03:44:40,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:40,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d013e2fa3d7693ba840a HTTP/1.1" 200 None 2026-05-02 03:44:40,179 - lava_callback - DEBUG - Processing node 69f2d013e2fa3d7693ba840a: baseline-arm-baylibre 2026-05-02 03:44:40,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d013e2fa3d7693ba840a 2026-05-02 03:44:40,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:40,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d013e2fa3d7693ba840a HTTP/1.1" 200 None 2026-05-02 03:44:40,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:40,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:44:40,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:44:40,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d013e2fa3d7693ba840a HTTP/1.1" 200 None 2026-05-02 03:44:40,233 - lava_callback - ERROR - Error processing callback for node 69f2d013e2fa3d7693ba840a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35058 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:45:21,055 - lava_callback - INFO - Received callback for node 69f2d03be2fa3d7693ba841a INFO: 172.20.0.1:48652 - "POST /node/69f2d03be2fa3d7693ba841a HTTP/1.0" 202 Accepted 2026-05-02 03:45:21,064 - lava_callback - INFO - Starting async job submit for node 69f2d03be2fa3d7693ba841a 2026-05-02 03:45:21,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:21,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d03be2fa3d7693ba841a HTTP/1.1" 200 None 2026-05-02 03:45:21,080 - lava_callback - DEBUG - Processing node 69f2d03be2fa3d7693ba841a: baseline-arm-clabbe 2026-05-02 03:45:21,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d03be2fa3d7693ba841a 2026-05-02 03:45:21,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:21,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d03be2fa3d7693ba841a HTTP/1.1" 200 None 2026-05-02 03:45:21,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:21,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:45:21,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:21,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d03be2fa3d7693ba841a HTTP/1.1" 200 None 2026-05-02 03:45:21,130 - lava_callback - ERROR - Error processing callback for node 69f2d03be2fa3d7693ba841a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47532 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:45:40,113 - lava_callback - INFO - Received callback for node 69f2d05ce2fa3d7693ba8423 INFO: 172.20.0.1:48806 - "POST /node/69f2d05ce2fa3d7693ba8423 HTTP/1.0" 202 Accepted 2026-05-02 03:45:40,119 - lava_callback - INFO - Starting async job submit for node 69f2d05ce2fa3d7693ba8423 2026-05-02 03:45:40,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:40,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d05ce2fa3d7693ba8423 HTTP/1.1" 200 None 2026-05-02 03:45:40,138 - lava_callback - DEBUG - Processing node 69f2d05ce2fa3d7693ba8423: baseline-arm-baylibre 2026-05-02 03:45:40,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d05ce2fa3d7693ba8423 2026-05-02 03:45:40,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:40,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d05ce2fa3d7693ba8423 HTTP/1.1" 200 None 2026-05-02 03:45:40,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:40,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:45:40,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:45:40,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d05ce2fa3d7693ba8423 HTTP/1.1" 200 None 2026-05-02 03:45:40,188 - lava_callback - ERROR - Error processing callback for node 69f2d05ce2fa3d7693ba8423: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59402 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:46:20,205 - lava_callback - INFO - Received callback for node 69f2d082e2fa3d7693ba8431 INFO: 172.20.0.1:59484 - "POST /node/69f2d082e2fa3d7693ba8431 HTTP/1.0" 202 Accepted 2026-05-02 03:46:20,211 - lava_callback - INFO - Starting async job submit for node 69f2d082e2fa3d7693ba8431 2026-05-02 03:46:20,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:46:20,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d082e2fa3d7693ba8431 HTTP/1.1" 200 None 2026-05-02 03:46:20,227 - lava_callback - DEBUG - Processing node 69f2d082e2fa3d7693ba8431: baseline-arm-baylibre 2026-05-02 03:46:20,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d082e2fa3d7693ba8431 2026-05-02 03:46:20,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:46:20,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d082e2fa3d7693ba8431 HTTP/1.1" 200 None 2026-05-02 03:46:20,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:46:20,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:46:20,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:46:20,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d082e2fa3d7693ba8431 HTTP/1.1" 200 None 2026-05-02 03:46:20,285 - lava_callback - ERROR - Error processing callback for node 69f2d082e2fa3d7693ba8431: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:47:01,190 - lava_callback - INFO - Received callback for node 69f2d0a9e2fa3d7693ba843c INFO: 172.20.0.1:40068 - "POST /node/69f2d0a9e2fa3d7693ba843c HTTP/1.0" 202 Accepted 2026-05-02 03:47:01,196 - lava_callback - INFO - Starting async job submit for node 69f2d0a9e2fa3d7693ba843c 2026-05-02 03:47:01,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:01,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d0a9e2fa3d7693ba843c HTTP/1.1" 200 None 2026-05-02 03:47:01,212 - lava_callback - DEBUG - Processing node 69f2d0a9e2fa3d7693ba843c: baseline-arm-clabbe 2026-05-02 03:47:01,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d0a9e2fa3d7693ba843c 2026-05-02 03:47:01,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:01,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d0a9e2fa3d7693ba843c HTTP/1.1" 200 None 2026-05-02 03:47:01,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:01,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:47:01,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:01,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d0a9e2fa3d7693ba843c HTTP/1.1" 200 None 2026-05-02 03:47:01,271 - lava_callback - ERROR - Error processing callback for node 69f2d0a9e2fa3d7693ba843c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:47:41,082 - lava_callback - INFO - Received callback for node 69f2d0cfe2fa3d7693ba8449 INFO: 172.20.0.1:42870 - "POST /node/69f2d0cfe2fa3d7693ba8449 HTTP/1.0" 202 Accepted 2026-05-02 03:47:41,088 - lava_callback - INFO - Starting async job submit for node 69f2d0cfe2fa3d7693ba8449 2026-05-02 03:47:41,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:41,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d0cfe2fa3d7693ba8449 HTTP/1.1" 200 None 2026-05-02 03:47:41,104 - lava_callback - DEBUG - Processing node 69f2d0cfe2fa3d7693ba8449: baseline-arm-clabbe 2026-05-02 03:47:41,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d0cfe2fa3d7693ba8449 2026-05-02 03:47:41,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:41,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d0cfe2fa3d7693ba8449 HTTP/1.1" 200 None 2026-05-02 03:47:41,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:41,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:47:41,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:47:41,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d0cfe2fa3d7693ba8449 HTTP/1.1" 200 None 2026-05-02 03:47:41,157 - lava_callback - ERROR - Error processing callback for node 69f2d0cfe2fa3d7693ba8449: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36656 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:48:20,342 - lava_callback - INFO - Received callback for node 69f2d0f1e2fa3d7693ba8455 INFO: 172.20.0.1:45332 - "POST /node/69f2d0f1e2fa3d7693ba8455 HTTP/1.0" 202 Accepted 2026-05-02 03:48:20,347 - lava_callback - INFO - Starting async job submit for node 69f2d0f1e2fa3d7693ba8455 2026-05-02 03:48:20,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:48:20,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d0f1e2fa3d7693ba8455 HTTP/1.1" 200 None 2026-05-02 03:48:20,362 - lava_callback - DEBUG - Processing node 69f2d0f1e2fa3d7693ba8455: baseline-arm-baylibre 2026-05-02 03:48:20,362 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d0f1e2fa3d7693ba8455 2026-05-02 03:48:20,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:48:20,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d0f1e2fa3d7693ba8455 HTTP/1.1" 200 None 2026-05-02 03:48:20,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:48:20,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:48:20,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:48:20,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d0f1e2fa3d7693ba8455 HTTP/1.1" 200 None 2026-05-02 03:48:20,408 - lava_callback - ERROR - Error processing callback for node 69f2d0f1e2fa3d7693ba8455: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42624 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:49:01,283 - lava_callback - INFO - Received callback for node 69f2d118e2fa3d7693ba8462 INFO: 172.20.0.1:59758 - "POST /node/69f2d118e2fa3d7693ba8462 HTTP/1.0" 202 Accepted 2026-05-02 03:49:01,288 - lava_callback - INFO - Starting async job submit for node 69f2d118e2fa3d7693ba8462 2026-05-02 03:49:01,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:01,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d118e2fa3d7693ba8462 HTTP/1.1" 200 None 2026-05-02 03:49:01,303 - lava_callback - DEBUG - Processing node 69f2d118e2fa3d7693ba8462: baseline-arm-clabbe 2026-05-02 03:49:01,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d118e2fa3d7693ba8462 2026-05-02 03:49:01,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:01,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d118e2fa3d7693ba8462 HTTP/1.1" 200 None 2026-05-02 03:49:01,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:01,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:49:01,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:01,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d118e2fa3d7693ba8462 HTTP/1.1" 200 None 2026-05-02 03:49:01,359 - lava_callback - ERROR - Error processing callback for node 69f2d118e2fa3d7693ba8462: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39562 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:49:41,278 - lava_callback - INFO - Received callback for node 69f2d13fe2fa3d7693ba846e INFO: 172.20.0.1:60186 - "POST /node/69f2d13fe2fa3d7693ba846e HTTP/1.0" 202 Accepted 2026-05-02 03:49:41,284 - lava_callback - INFO - Starting async job submit for node 69f2d13fe2fa3d7693ba846e 2026-05-02 03:49:41,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:41,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d13fe2fa3d7693ba846e HTTP/1.1" 200 None 2026-05-02 03:49:41,299 - lava_callback - DEBUG - Processing node 69f2d13fe2fa3d7693ba846e: baseline-arm-clabbe 2026-05-02 03:49:41,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d13fe2fa3d7693ba846e 2026-05-02 03:49:41,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:41,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d13fe2fa3d7693ba846e HTTP/1.1" 200 None 2026-05-02 03:49:41,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:41,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:49:41,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:41,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d13fe2fa3d7693ba846e HTTP/1.1" 200 None 2026-05-02 03:49:41,345 - lava_callback - ERROR - Error processing callback for node 69f2d13fe2fa3d7693ba846e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60188 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:49:51,653 - lava_callback - INFO - Received callback for node 69f572aeed4ed40b02241b48 INFO: 172.20.0.1:55546 - "POST /node/69f572aeed4ed40b02241b48 HTTP/1.0" 202 Accepted 2026-05-02 03:49:51,660 - lava_callback - INFO - Starting async job submit for node 69f572aeed4ed40b02241b48 2026-05-02 03:49:51,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:51,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f572aeed4ed40b02241b48 HTTP/1.1" 200 None 2026-05-02 03:49:51,725 - lava_callback - DEBUG - Processing node 69f572aeed4ed40b02241b48: baseline-arm64 2026-05-02 03:49:52,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:49:52,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:49:52,288 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f572aeed4ed40b02241b48/log.txt.gz 2026-05-02 03:49:52,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:49:52,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:49:52,692 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f572aeed4ed40b02241b48/lava-logs.yaml 2026-05-02 03:49:52,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 03:49:52,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 03:49:52,832 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f572aeed4ed40b02241b48/lava_callback.json.gz DEBUG: Setup failed for 69f572aeed4ed40b02241b48. Transitting job node result: incomplete -> fail 2026-05-02 03:49:52,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:52,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f572aeed4ed40b02241b48 HTTP/1.1" 200 None 2026-05-02 03:49:52,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:52,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 03:49:52,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:52,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f572aeed4ed40b02241b48 HTTP/1.1" 200 None 2026-05-02 03:49:53,043 - lava_callback - INFO - Completed processing callback for node 69f572aeed4ed40b02241b48 2026-05-02 03:49:53,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:49:53,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48542 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:50:24,433 - lava_callback - INFO - Received callback for node 69f2d16ae2fa3d7693ba847f INFO: 172.20.0.1:48548 - "POST /node/69f2d16ae2fa3d7693ba847f HTTP/1.0" 202 Accepted 2026-05-02 03:50:24,439 - lava_callback - INFO - Starting async job submit for node 69f2d16ae2fa3d7693ba847f 2026-05-02 03:50:24,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:50:24,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d16ae2fa3d7693ba847f HTTP/1.1" 200 None 2026-05-02 03:50:24,455 - lava_callback - DEBUG - Processing node 69f2d16ae2fa3d7693ba847f: baseline-arm64-baylibre 2026-05-02 03:50:24,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d16ae2fa3d7693ba847f 2026-05-02 03:50:24,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:50:24,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d16ae2fa3d7693ba847f HTTP/1.1" 200 None 2026-05-02 03:50:24,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:50:24,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 03:50:24,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:50:24,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d16ae2fa3d7693ba847f HTTP/1.1" 200 None 2026-05-02 03:50:24,499 - lava_callback - ERROR - Error processing callback for node 69f2d16ae2fa3d7693ba847f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47274 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:52:40,527 - lava_callback - INFO - Received callback for node 69f2d1f4e2fa3d7693ba84b9 INFO: 172.20.0.1:40414 - "POST /node/69f2d1f4e2fa3d7693ba84b9 HTTP/1.0" 202 Accepted 2026-05-02 03:52:40,533 - lava_callback - INFO - Starting async job submit for node 69f2d1f4e2fa3d7693ba84b9 2026-05-02 03:52:40,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:52:40,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d1f4e2fa3d7693ba84b9 HTTP/1.1" 200 None 2026-05-02 03:52:40,548 - lava_callback - DEBUG - Processing node 69f2d1f4e2fa3d7693ba84b9: baseline-arm-baylibre 2026-05-02 03:52:40,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d1f4e2fa3d7693ba84b9 2026-05-02 03:52:40,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:52:40,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d1f4e2fa3d7693ba84b9 HTTP/1.1" 200 None 2026-05-02 03:52:40,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:52:40,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2023113aba9734a494540 HTTP/1.1" 200 None 2026-05-02 03:52:40,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:52:40,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d1f4e2fa3d7693ba84b9 HTTP/1.1" 200 None 2026-05-02 03:52:40,593 - lava_callback - ERROR - Error processing callback for node 69f2d1f4e2fa3d7693ba84b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58406 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:53:00,577 - lava_callback - INFO - Received callback for node 69f2d219e2fa3d7693ba84c8 INFO: 172.20.0.1:40288 - "POST /node/69f2d219e2fa3d7693ba84c8 HTTP/1.0" 202 Accepted 2026-05-02 03:53:00,582 - lava_callback - INFO - Starting async job submit for node 69f2d219e2fa3d7693ba84c8 2026-05-02 03:53:00,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:53:00,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d219e2fa3d7693ba84c8 HTTP/1.1" 200 None 2026-05-02 03:53:00,598 - lava_callback - DEBUG - Processing node 69f2d219e2fa3d7693ba84c8: baseline-arm-baylibre 2026-05-02 03:53:00,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d219e2fa3d7693ba84c8 2026-05-02 03:53:00,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:53:00,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d219e2fa3d7693ba84c8 HTTP/1.1" 200 None 2026-05-02 03:53:00,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:53:00,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2023113aba9734a494540 HTTP/1.1" 200 None 2026-05-02 03:53:00,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:53:00,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d219e2fa3d7693ba84c8 HTTP/1.1" 200 None 2026-05-02 03:53:00,647 - lava_callback - ERROR - Error processing callback for node 69f2d219e2fa3d7693ba84c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41742 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:54:00,629 - lava_callback - INFO - Received callback for node 69f2d249e2fa3d7693ba84d7 INFO: 172.20.0.1:51720 - "POST /node/69f2d249e2fa3d7693ba84d7 HTTP/1.0" 202 Accepted 2026-05-02 03:54:00,635 - lava_callback - INFO - Starting async job submit for node 69f2d249e2fa3d7693ba84d7 2026-05-02 03:54:00,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:54:00,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d249e2fa3d7693ba84d7 HTTP/1.1" 200 None 2026-05-02 03:54:00,654 - lava_callback - DEBUG - Processing node 69f2d249e2fa3d7693ba84d7: baseline-arm64-baylibre 2026-05-02 03:54:00,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d249e2fa3d7693ba84d7 2026-05-02 03:54:00,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:54:00,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d249e2fa3d7693ba84d7 HTTP/1.1" 200 None 2026-05-02 03:54:00,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:54:00,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 03:54:00,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:54:00,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d249e2fa3d7693ba84d7 HTTP/1.1" 200 None 2026-05-02 03:54:00,700 - lava_callback - ERROR - Error processing callback for node 69f2d249e2fa3d7693ba84d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:55:00,637 - lava_callback - INFO - Received callback for node 69f2d28ae2fa3d7693ba84fa INFO: 172.20.0.1:48428 - "POST /node/69f2d28ae2fa3d7693ba84fa HTTP/1.0" 202 Accepted 2026-05-02 03:55:00,646 - lava_callback - INFO - Starting async job submit for node 69f2d28ae2fa3d7693ba84fa 2026-05-02 03:55:00,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:00,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d28ae2fa3d7693ba84fa HTTP/1.1" 200 None 2026-05-02 03:55:00,661 - lava_callback - DEBUG - Processing node 69f2d28ae2fa3d7693ba84fa: baseline-arm-baylibre 2026-05-02 03:55:00,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d28ae2fa3d7693ba84fa 2026-05-02 03:55:00,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:00,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d28ae2fa3d7693ba84fa HTTP/1.1" 200 None 2026-05-02 03:55:00,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:00,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2023113aba9734a494540 HTTP/1.1" 200 None 2026-05-02 03:55:00,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:00,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d28ae2fa3d7693ba84fa HTTP/1.1" 200 None 2026-05-02 03:55:00,712 - lava_callback - ERROR - Error processing callback for node 69f2d28ae2fa3d7693ba84fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57828 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:55:40,777 - lava_callback - INFO - Received callback for node 69f2d2b7e2fa3d7693ba850c INFO: 172.20.0.1:53700 - "POST /node/69f2d2b7e2fa3d7693ba850c HTTP/1.0" 202 Accepted 2026-05-02 03:55:40,782 - lava_callback - INFO - Starting async job submit for node 69f2d2b7e2fa3d7693ba850c 2026-05-02 03:55:40,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:40,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d2b7e2fa3d7693ba850c HTTP/1.1" 200 None 2026-05-02 03:55:40,798 - lava_callback - DEBUG - Processing node 69f2d2b7e2fa3d7693ba850c: baseline-arm64-baylibre 2026-05-02 03:55:40,798 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d2b7e2fa3d7693ba850c 2026-05-02 03:55:40,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:40,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d2b7e2fa3d7693ba850c HTTP/1.1" 200 None 2026-05-02 03:55:40,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:40,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 03:55:40,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:55:40,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d2b7e2fa3d7693ba850c HTTP/1.1" 200 None 2026-05-02 03:55:40,849 - lava_callback - ERROR - Error processing callback for node 69f2d2b7e2fa3d7693ba850c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49430 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:57:00,799 - lava_callback - INFO - Received callback for node 69f2d2f8e2fa3d7693ba852e INFO: 172.20.0.1:49302 - "POST /node/69f2d2f8e2fa3d7693ba852e HTTP/1.0" 202 Accepted 2026-05-02 03:57:00,805 - lava_callback - INFO - Starting async job submit for node 69f2d2f8e2fa3d7693ba852e 2026-05-02 03:57:00,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:00,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d2f8e2fa3d7693ba852e HTTP/1.1" 200 None 2026-05-02 03:57:00,825 - lava_callback - DEBUG - Processing node 69f2d2f8e2fa3d7693ba852e: baseline-arm-baylibre 2026-05-02 03:57:00,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d2f8e2fa3d7693ba852e 2026-05-02 03:57:00,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:00,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d2f8e2fa3d7693ba852e HTTP/1.1" 200 None 2026-05-02 03:57:00,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:00,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2023113aba9734a494540 HTTP/1.1" 200 None 2026-05-02 03:57:00,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:00,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d2f8e2fa3d7693ba852e HTTP/1.1" 200 None 2026-05-02 03:57:00,894 - lava_callback - ERROR - Error processing callback for node 69f2d2f8e2fa3d7693ba852e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:57:41,665 - lava_callback - INFO - Received callback for node 69f2d320e2fa3d7693ba853f INFO: 172.20.0.1:59388 - "POST /node/69f2d320e2fa3d7693ba853f HTTP/1.0" 202 Accepted 2026-05-02 03:57:41,670 - lava_callback - INFO - Starting async job submit for node 69f2d320e2fa3d7693ba853f 2026-05-02 03:57:41,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:41,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d320e2fa3d7693ba853f HTTP/1.1" 200 None 2026-05-02 03:57:41,686 - lava_callback - DEBUG - Processing node 69f2d320e2fa3d7693ba853f: baseline-arm-clabbe 2026-05-02 03:57:41,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d320e2fa3d7693ba853f 2026-05-02 03:57:41,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:41,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d320e2fa3d7693ba853f HTTP/1.1" 200 None 2026-05-02 03:57:41,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:41,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5da98305534e972331c0 HTTP/1.1" 200 None 2026-05-02 03:57:41,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:57:41,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d320e2fa3d7693ba853f HTTP/1.1" 200 None 2026-05-02 03:57:41,732 - lava_callback - ERROR - Error processing callback for node 69f2d320e2fa3d7693ba853f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59994 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:58:00,830 - lava_callback - INFO - Received callback for node 69f2d343e2fa3d7693ba854d INFO: 172.20.0.1:44834 - "POST /node/69f2d343e2fa3d7693ba854d HTTP/1.0" 202 Accepted 2026-05-02 03:58:00,835 - lava_callback - INFO - Starting async job submit for node 69f2d343e2fa3d7693ba854d 2026-05-02 03:58:00,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:00,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d343e2fa3d7693ba854d HTTP/1.1" 200 None 2026-05-02 03:58:00,850 - lava_callback - DEBUG - Processing node 69f2d343e2fa3d7693ba854d: baseline-arm-baylibre 2026-05-02 03:58:00,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d343e2fa3d7693ba854d 2026-05-02 03:58:00,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:00,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d343e2fa3d7693ba854d HTTP/1.1" 200 None 2026-05-02 03:58:00,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:00,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2023113aba9734a494540 HTTP/1.1" 200 None 2026-05-02 03:58:00,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:00,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d343e2fa3d7693ba854d HTTP/1.1" 200 None 2026-05-02 03:58:00,902 - lava_callback - ERROR - Error processing callback for node 69f2d343e2fa3d7693ba854d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:58:40,895 - lava_callback - INFO - Received callback for node 69f2d367e2fa3d7693ba855a INFO: 172.20.0.1:56458 - "POST /node/69f2d367e2fa3d7693ba855a HTTP/1.0" 202 Accepted 2026-05-02 03:58:40,901 - lava_callback - INFO - Starting async job submit for node 69f2d367e2fa3d7693ba855a 2026-05-02 03:58:40,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:40,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d367e2fa3d7693ba855a HTTP/1.1" 200 None 2026-05-02 03:58:40,916 - lava_callback - DEBUG - Processing node 69f2d367e2fa3d7693ba855a: baseline-arm-baylibre 2026-05-02 03:58:40,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d367e2fa3d7693ba855a 2026-05-02 03:58:40,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:40,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d367e2fa3d7693ba855a HTTP/1.1" 200 None 2026-05-02 03:58:40,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:40,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2023113aba9734a494540 HTTP/1.1" 200 None 2026-05-02 03:58:40,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:58:40,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d367e2fa3d7693ba855a HTTP/1.1" 200 None 2026-05-02 03:58:40,971 - lava_callback - ERROR - Error processing callback for node 69f2d367e2fa3d7693ba855a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43268 - "GET / HTTP/1.0" 200 OK 2026-05-02 03:59:20,862 - lava_callback - INFO - Received callback for node 69f2d393e2fa3d7693ba856a INFO: 172.20.0.1:41722 - "POST /node/69f2d393e2fa3d7693ba856a HTTP/1.0" 202 Accepted 2026-05-02 03:59:20,868 - lava_callback - INFO - Starting async job submit for node 69f2d393e2fa3d7693ba856a 2026-05-02 03:59:20,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:59:20,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d393e2fa3d7693ba856a HTTP/1.1" 200 None 2026-05-02 03:59:20,885 - lava_callback - DEBUG - Processing node 69f2d393e2fa3d7693ba856a: baseline-arm64-baylibre 2026-05-02 03:59:20,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d393e2fa3d7693ba856a 2026-05-02 03:59:20,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:59:20,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d393e2fa3d7693ba856a HTTP/1.1" 200 None 2026-05-02 03:59:20,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:59:20,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 03:59:20,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 03:59:20,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d393e2fa3d7693ba856a HTTP/1.1" 200 None 2026-05-02 03:59:20,932 - lava_callback - ERROR - Error processing callback for node 69f2d393e2fa3d7693ba856a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46364 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:01:21,259 - lava_callback - INFO - Received callback for node 69f2d400e2fa3d7693ba85a1 INFO: 172.20.0.1:45094 - "POST /node/69f2d400e2fa3d7693ba85a1 HTTP/1.0" 202 Accepted 2026-05-02 04:01:21,265 - lava_callback - INFO - Starting async job submit for node 69f2d400e2fa3d7693ba85a1 2026-05-02 04:01:21,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:21,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d400e2fa3d7693ba85a1 HTTP/1.1" 200 None 2026-05-02 04:01:21,279 - lava_callback - DEBUG - Processing node 69f2d400e2fa3d7693ba85a1: baseline-arm64-baylibre 2026-05-02 04:01:21,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d400e2fa3d7693ba85a1 2026-05-02 04:01:21,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:21,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d400e2fa3d7693ba85a1 HTTP/1.1" 200 None 2026-05-02 04:01:21,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:21,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:01:21,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:21,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d400e2fa3d7693ba85a1 HTTP/1.1" 200 None 2026-05-02 04:01:21,323 - lava_callback - ERROR - Error processing callback for node 69f2d400e2fa3d7693ba85a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45108 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:01:32,736 - lava_callback - INFO - Received callback for node 69f572f2ed4ed40b02241b65 INFO: 172.20.0.1:36552 - "POST /node/69f572f2ed4ed40b02241b65 HTTP/1.0" 202 Accepted 2026-05-02 04:01:32,743 - lava_callback - INFO - Starting async job submit for node 69f572f2ed4ed40b02241b65 2026-05-02 04:01:32,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:32,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f572f2ed4ed40b02241b65 HTTP/1.1" 200 None 2026-05-02 04:01:32,814 - lava_callback - DEBUG - Processing node 69f572f2ed4ed40b02241b65: ltp-smoke 2026-05-02 04:01:33,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:01:33,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:01:33,514 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f572f2ed4ed40b02241b65/log.txt.gz 2026-05-02 04:01:33,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:01:33,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:01:33,981 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f572f2ed4ed40b02241b65/lava-logs.yaml 2026-05-02 04:01:33,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:01:34,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:01:34,104 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f572f2ed4ed40b02241b65/lava_callback.json.gz 2026-05-02 04:01:34,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:34,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f572f2ed4ed40b02241b65 HTTP/1.1" 200 None 2026-05-02 04:01:34,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:34,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54464ed4ed40b0223db05 HTTP/1.1" 200 None 2026-05-02 04:01:34,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:34,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f572f2ed4ed40b02241b65 HTTP/1.1" 200 None 2026-05-02 04:01:34,400 - lava_callback - INFO - Completed processing callback for node 69f572f2ed4ed40b02241b65 INFO: 172.20.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52046 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:01:53,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:01:53,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58222 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:03:01,089 - lava_callback - INFO - Received callback for node 69f2d46ce2fa3d7693ba85d6 INFO: 172.20.0.1:44684 - "POST /node/69f2d46ce2fa3d7693ba85d6 HTTP/1.0" 202 Accepted 2026-05-02 04:03:01,094 - lava_callback - INFO - Starting async job submit for node 69f2d46ce2fa3d7693ba85d6 2026-05-02 04:03:01,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:03:01,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d46ce2fa3d7693ba85d6 HTTP/1.1" 200 None 2026-05-02 04:03:01,114 - lava_callback - DEBUG - Processing node 69f2d46ce2fa3d7693ba85d6: baseline-arm64-baylibre 2026-05-02 04:03:01,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d46ce2fa3d7693ba85d6 2026-05-02 04:03:01,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:03:01,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d46ce2fa3d7693ba85d6 HTTP/1.1" 200 None 2026-05-02 04:03:01,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:03:01,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:03:01,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:03:01,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d46ce2fa3d7693ba85d6 HTTP/1.1" 200 None 2026-05-02 04:03:01,164 - lava_callback - ERROR - Error processing callback for node 69f2d46ce2fa3d7693ba85d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:04:21,247 - lava_callback - INFO - Received callback for node 69f2d4bae2fa3d7693ba85fe INFO: 172.20.0.1:46894 - "POST /node/69f2d4bae2fa3d7693ba85fe HTTP/1.0" 202 Accepted 2026-05-02 04:04:21,253 - lava_callback - INFO - Starting async job submit for node 69f2d4bae2fa3d7693ba85fe 2026-05-02 04:04:21,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:04:21,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d4bae2fa3d7693ba85fe HTTP/1.1" 200 None 2026-05-02 04:04:21,267 - lava_callback - DEBUG - Processing node 69f2d4bae2fa3d7693ba85fe: baseline-arm64-baylibre 2026-05-02 04:04:21,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d4bae2fa3d7693ba85fe 2026-05-02 04:04:21,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:04:21,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d4bae2fa3d7693ba85fe HTTP/1.1" 200 None 2026-05-02 04:04:21,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:04:21,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:04:21,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:04:21,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d4bae2fa3d7693ba85fe HTTP/1.1" 200 None 2026-05-02 04:04:21,324 - lava_callback - ERROR - Error processing callback for node 69f2d4bae2fa3d7693ba85fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48924 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:05:41,248 - lava_callback - INFO - Received callback for node 69f2d502e2fa3d7693ba8621 INFO: 172.20.0.1:46174 - "POST /node/69f2d502e2fa3d7693ba8621 HTTP/1.0" 202 Accepted 2026-05-02 04:05:41,253 - lava_callback - INFO - Starting async job submit for node 69f2d502e2fa3d7693ba8621 2026-05-02 04:05:41,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:05:41,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d502e2fa3d7693ba8621 HTTP/1.1" 200 None 2026-05-02 04:05:41,269 - lava_callback - DEBUG - Processing node 69f2d502e2fa3d7693ba8621: baseline-arm64-baylibre 2026-05-02 04:05:41,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d502e2fa3d7693ba8621 2026-05-02 04:05:41,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:05:41,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d502e2fa3d7693ba8621 HTTP/1.1" 200 None 2026-05-02 04:05:41,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:05:41,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:05:41,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:05:41,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d502e2fa3d7693ba8621 HTTP/1.1" 200 None 2026-05-02 04:05:41,314 - lava_callback - ERROR - Error processing callback for node 69f2d502e2fa3d7693ba8621: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58020 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:07:02,125 - lava_callback - INFO - Received callback for node 69f2d559e2fa3d7693ba8651 INFO: 172.20.0.1:57346 - "POST /node/69f2d559e2fa3d7693ba8651 HTTP/1.0" 202 Accepted 2026-05-02 04:07:02,130 - lava_callback - INFO - Starting async job submit for node 69f2d559e2fa3d7693ba8651 2026-05-02 04:07:02,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:07:02,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d559e2fa3d7693ba8651 HTTP/1.1" 200 None 2026-05-02 04:07:02,146 - lava_callback - DEBUG - Processing node 69f2d559e2fa3d7693ba8651: baseline-arm64-clabbe 2026-05-02 04:07:02,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d559e2fa3d7693ba8651 2026-05-02 04:07:02,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:07:02,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d559e2fa3d7693ba8651 HTTP/1.1" 200 None 2026-05-02 04:07:02,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:07:02,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:07:02,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:07:02,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d559e2fa3d7693ba8651 HTTP/1.1" 200 None 2026-05-02 04:07:02,200 - lava_callback - ERROR - Error processing callback for node 69f2d559e2fa3d7693ba8651: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:08:21,394 - lava_callback - INFO - Received callback for node 69f2d5a1e2fa3d7693ba8682 INFO: 172.20.0.1:51420 - "POST /node/69f2d5a1e2fa3d7693ba8682 HTTP/1.0" 202 Accepted 2026-05-02 04:08:21,400 - lava_callback - INFO - Starting async job submit for node 69f2d5a1e2fa3d7693ba8682 2026-05-02 04:08:21,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:21,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d5a1e2fa3d7693ba8682 HTTP/1.1" 200 None 2026-05-02 04:08:21,417 - lava_callback - DEBUG - Processing node 69f2d5a1e2fa3d7693ba8682: baseline-arm64-baylibre 2026-05-02 04:08:21,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d5a1e2fa3d7693ba8682 2026-05-02 04:08:21,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:21,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d5a1e2fa3d7693ba8682 HTTP/1.1" 200 None 2026-05-02 04:08:21,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:21,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:08:21,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:21,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d5a1e2fa3d7693ba8682 HTTP/1.1" 200 None 2026-05-02 04:08:21,462 - lava_callback - ERROR - Error processing callback for node 69f2d5a1e2fa3d7693ba8682: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51430 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:08:34,995 - lava_callback - INFO - Received callback for node 69f574faed4ed40b02241c49 INFO: 172.20.0.1:56046 - "POST /node/69f574faed4ed40b02241c49 HTTP/1.0" 202 Accepted 2026-05-02 04:08:35,003 - lava_callback - INFO - Starting async job submit for node 69f574faed4ed40b02241c49 2026-05-02 04:08:35,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:35,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f574faed4ed40b02241c49 HTTP/1.1" 200 None 2026-05-02 04:08:35,097 - lava_callback - DEBUG - Processing node 69f574faed4ed40b02241c49: baseline-arm 2026-05-02 04:08:35,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:08:35,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:08:35,829 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f574faed4ed40b02241c49/log.txt.gz 2026-05-02 04:08:36,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:08:36,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:08:36,353 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f574faed4ed40b02241c49/lava-logs.yaml 2026-05-02 04:08:36,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:08:36,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:08:36,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f574faed4ed40b02241c49/lava_callback.json.gz INFO: 172.20.0.1:56060 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 69f574faed4ed40b02241c49. Transitting job node result: incomplete -> fail 2026-05-02 04:08:36,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:36,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f574faed4ed40b02241c49 HTTP/1.1" 200 None 2026-05-02 04:08:36,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:36,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 04:08:36,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:36,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f574faed4ed40b02241c49 HTTP/1.1" 200 None 2026-05-02 04:08:36,820 - lava_callback - INFO - Completed processing callback for node 69f574faed4ed40b02241c49 INFO: 172.20.0.1:47558 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:08:53,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:08:53,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:09:21,368 - lava_callback - INFO - Received callback for node 69f55e94ed4ed40b022411f2 INFO: 172.20.0.1:36336 - "POST /node/69f55e94ed4ed40b022411f2 HTTP/1.0" 202 Accepted 2026-05-02 04:09:21,375 - lava_callback - INFO - Starting async job submit for node 69f55e94ed4ed40b022411f2 2026-05-02 04:09:21,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:21,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55e94ed4ed40b022411f2 HTTP/1.1" 200 None 2026-05-02 04:09:21,428 - lava_callback - DEBUG - Processing node 69f55e94ed4ed40b022411f2: baseline-arm64 2026-05-02 04:09:22,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:09:22,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:09:22,160 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55e94ed4ed40b022411f2/log.txt.gz 2026-05-02 04:09:22,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:09:22,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:09:22,558 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55e94ed4ed40b022411f2/lava-logs.yaml 2026-05-02 04:09:22,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:09:22,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:09:22,693 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f55e94ed4ed40b022411f2/lava_callback.json.gz 2026-05-02 04:09:22,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:22,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55e94ed4ed40b022411f2 HTTP/1.1" 200 None 2026-05-02 04:09:22,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:22,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 04:09:22,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:22,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55e94ed4ed40b022411f2 HTTP/1.1" 200 None 2026-05-02 04:09:22,809 - lava_callback - INFO - Completed processing callback for node 69f55e94ed4ed40b022411f2 2026-05-02 04:09:23,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:23,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:09:42,243 - lava_callback - INFO - Received callback for node 69f2d5f5e2fa3d7693ba86b8 INFO: 172.20.0.1:55716 - "POST /node/69f2d5f5e2fa3d7693ba86b8 HTTP/1.0" 202 Accepted 2026-05-02 04:09:42,251 - lava_callback - INFO - Starting async job submit for node 69f2d5f5e2fa3d7693ba86b8 2026-05-02 04:09:42,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:42,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d5f5e2fa3d7693ba86b8 HTTP/1.1" 200 None 2026-05-02 04:09:42,266 - lava_callback - DEBUG - Processing node 69f2d5f5e2fa3d7693ba86b8: baseline-arm64-clabbe 2026-05-02 04:09:42,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d5f5e2fa3d7693ba86b8 2026-05-02 04:09:42,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:42,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d5f5e2fa3d7693ba86b8 HTTP/1.1" 200 None 2026-05-02 04:09:42,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:42,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:09:42,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:09:42,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d5f5e2fa3d7693ba86b8 HTTP/1.1" 200 None 2026-05-02 04:09:42,315 - lava_callback - ERROR - Error processing callback for node 69f2d5f5e2fa3d7693ba86b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51052 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:10:14,399 - lava_callback - INFO - Received callback for node 69f57850ed4ed40b02241db5 INFO: 172.20.0.1:54290 - "POST /node/69f57850ed4ed40b02241db5 HTTP/1.0" 202 Accepted 2026-05-02 04:10:14,405 - lava_callback - INFO - Starting async job submit for node 69f57850ed4ed40b02241db5 2026-05-02 04:10:14,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:14,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f57850ed4ed40b02241db5 HTTP/1.1" 200 None 2026-05-02 04:10:14,437 - lava_callback - DEBUG - Processing node 69f57850ed4ed40b02241db5: baseline-arm64 2026-05-02 04:10:14,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:10:15,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:10:15,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f57850ed4ed40b02241db5/log.txt.gz INFO: 172.20.0.1:54292 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 04:10:15,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:10:15,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:10:15,354 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f57850ed4ed40b02241db5/lava-logs.yaml 2026-05-02 04:10:15,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:10:15,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:10:15,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f57850ed4ed40b02241db5/lava_callback.json.gz 2026-05-02 04:10:15,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:15,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f57850ed4ed40b02241db5 HTTP/1.1" 200 None 2026-05-02 04:10:15,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:15,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 04:10:15,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:15,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f57850ed4ed40b02241db5 HTTP/1.1" 200 None 2026-05-02 04:10:15,580 - lava_callback - INFO - Completed processing callback for node 69f57850ed4ed40b02241db5 INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:10:23,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:23,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51230 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:10:41,533 - lava_callback - INFO - Received callback for node 69f2d63de2fa3d7693ba86d7 INFO: 172.20.0.1:43672 - "POST /node/69f2d63de2fa3d7693ba86d7 HTTP/1.0" 202 Accepted 2026-05-02 04:10:41,542 - lava_callback - INFO - Starting async job submit for node 69f2d63de2fa3d7693ba86d7 2026-05-02 04:10:41,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:41,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d63de2fa3d7693ba86d7 HTTP/1.1" 200 None 2026-05-02 04:10:41,558 - lava_callback - DEBUG - Processing node 69f2d63de2fa3d7693ba86d7: baseline-arm64-baylibre 2026-05-02 04:10:41,558 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d63de2fa3d7693ba86d7 2026-05-02 04:10:41,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:41,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d63de2fa3d7693ba86d7 HTTP/1.1" 200 None 2026-05-02 04:10:41,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:41,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:10:41,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:10:41,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d63de2fa3d7693ba86d7 HTTP/1.1" 200 None 2026-05-02 04:10:41,606 - lava_callback - ERROR - Error processing callback for node 69f2d63de2fa3d7693ba86d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33674 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:12:22,359 - lava_callback - INFO - Received callback for node 69f2d691e2fa3d7693ba870e INFO: 172.20.0.1:36340 - "POST /node/69f2d691e2fa3d7693ba870e HTTP/1.0" 202 Accepted 2026-05-02 04:12:22,364 - lava_callback - INFO - Starting async job submit for node 69f2d691e2fa3d7693ba870e 2026-05-02 04:12:22,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:12:22,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d691e2fa3d7693ba870e HTTP/1.1" 200 None 2026-05-02 04:12:22,387 - lava_callback - DEBUG - Processing node 69f2d691e2fa3d7693ba870e: baseline-arm64-clabbe 2026-05-02 04:12:22,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d691e2fa3d7693ba870e 2026-05-02 04:12:22,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:12:22,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d691e2fa3d7693ba870e HTTP/1.1" 200 None 2026-05-02 04:12:22,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:12:22,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:12:22,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:12:22,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d691e2fa3d7693ba870e HTTP/1.1" 200 None 2026-05-02 04:12:22,432 - lava_callback - ERROR - Error processing callback for node 69f2d691e2fa3d7693ba870e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:13:21,656 - lava_callback - INFO - Received callback for node 69f2d6d9e2fa3d7693ba872e INFO: 172.20.0.1:47020 - "POST /node/69f2d6d9e2fa3d7693ba872e HTTP/1.0" 202 Accepted 2026-05-02 04:13:21,661 - lava_callback - INFO - Starting async job submit for node 69f2d6d9e2fa3d7693ba872e 2026-05-02 04:13:21,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:13:21,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d6d9e2fa3d7693ba872e HTTP/1.1" 200 None 2026-05-02 04:13:21,681 - lava_callback - DEBUG - Processing node 69f2d6d9e2fa3d7693ba872e: baseline-arm64-baylibre 2026-05-02 04:13:21,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d6d9e2fa3d7693ba872e 2026-05-02 04:13:21,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:13:21,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d6d9e2fa3d7693ba872e HTTP/1.1" 200 None 2026-05-02 04:13:21,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:13:21,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:13:21,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:13:21,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d6d9e2fa3d7693ba872e HTTP/1.1" 200 None 2026-05-02 04:13:21,758 - lava_callback - ERROR - Error processing callback for node 69f2d6d9e2fa3d7693ba872e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:14:42,434 - lava_callback - INFO - Received callback for node 69f2d72ce2fa3d7693ba875c INFO: 172.20.0.1:58422 - "POST /node/69f2d72ce2fa3d7693ba875c HTTP/1.0" 202 Accepted 2026-05-02 04:14:42,440 - lava_callback - INFO - Starting async job submit for node 69f2d72ce2fa3d7693ba875c 2026-05-02 04:14:42,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:14:42,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d72ce2fa3d7693ba875c HTTP/1.1" 200 None 2026-05-02 04:14:42,459 - lava_callback - DEBUG - Processing node 69f2d72ce2fa3d7693ba875c: baseline-arm64-clabbe 2026-05-02 04:14:42,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d72ce2fa3d7693ba875c 2026-05-02 04:14:42,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:14:42,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d72ce2fa3d7693ba875c HTTP/1.1" 200 None 2026-05-02 04:14:42,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:14:42,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:14:42,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:14:42,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d72ce2fa3d7693ba875c HTTP/1.1" 200 None 2026-05-02 04:14:42,511 - lava_callback - ERROR - Error processing callback for node 69f2d72ce2fa3d7693ba875c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60856 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:16:02,029 - lava_callback - INFO - Received callback for node 69f2d76ee2fa3d7693ba877e INFO: 172.20.0.1:34620 - "POST /node/69f2d76ee2fa3d7693ba877e HTTP/1.0" 202 Accepted 2026-05-02 04:16:02,034 - lava_callback - INFO - Starting async job submit for node 69f2d76ee2fa3d7693ba877e 2026-05-02 04:16:02,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:16:02,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d76ee2fa3d7693ba877e HTTP/1.1" 200 None 2026-05-02 04:16:02,052 - lava_callback - DEBUG - Processing node 69f2d76ee2fa3d7693ba877e: baseline-arm64-baylibre 2026-05-02 04:16:02,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d76ee2fa3d7693ba877e 2026-05-02 04:16:02,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:16:02,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d76ee2fa3d7693ba877e HTTP/1.1" 200 None 2026-05-02 04:16:02,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:16:02,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:16:02,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:16:02,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d76ee2fa3d7693ba877e HTTP/1.1" 200 None 2026-05-02 04:16:02,098 - lava_callback - ERROR - Error processing callback for node 69f2d76ee2fa3d7693ba877e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48204 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:17:22,508 - lava_callback - INFO - Received callback for node 69f2d7c1e2fa3d7693ba87ab INFO: 172.20.0.1:55578 - "POST /node/69f2d7c1e2fa3d7693ba87ab HTTP/1.0" 202 Accepted 2026-05-02 04:17:22,513 - lava_callback - INFO - Starting async job submit for node 69f2d7c1e2fa3d7693ba87ab 2026-05-02 04:17:22,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:17:22,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d7c1e2fa3d7693ba87ab HTTP/1.1" 200 None 2026-05-02 04:17:22,529 - lava_callback - DEBUG - Processing node 69f2d7c1e2fa3d7693ba87ab: baseline-arm64-clabbe 2026-05-02 04:17:22,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d7c1e2fa3d7693ba87ab 2026-05-02 04:17:22,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:17:22,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d7c1e2fa3d7693ba87ab HTTP/1.1" 200 None 2026-05-02 04:17:22,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:17:22,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:17:22,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:17:22,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d7c1e2fa3d7693ba87ab HTTP/1.1" 200 None 2026-05-02 04:17:22,590 - lava_callback - ERROR - Error processing callback for node 69f2d7c1e2fa3d7693ba87ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55088 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:18:22,030 - lava_callback - INFO - Received callback for node 69f2d808e2fa3d7693ba87ca INFO: 172.20.0.1:44082 - "POST /node/69f2d808e2fa3d7693ba87ca HTTP/1.0" 202 Accepted 2026-05-02 04:18:22,035 - lava_callback - INFO - Starting async job submit for node 69f2d808e2fa3d7693ba87ca 2026-05-02 04:18:22,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:18:22,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d808e2fa3d7693ba87ca HTTP/1.1" 200 None 2026-05-02 04:18:22,051 - lava_callback - DEBUG - Processing node 69f2d808e2fa3d7693ba87ca: baseline-arm64-baylibre 2026-05-02 04:18:22,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d808e2fa3d7693ba87ca 2026-05-02 04:18:22,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:18:22,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d808e2fa3d7693ba87ca HTTP/1.1" 200 None 2026-05-02 04:18:22,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:18:22,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:18:22,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:18:22,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d808e2fa3d7693ba87ca HTTP/1.1" 200 None 2026-05-02 04:18:22,106 - lava_callback - ERROR - Error processing callback for node 69f2d808e2fa3d7693ba87ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:19:41,976 - lava_callback - INFO - Received callback for node 69f2d852e2fa3d7693ba87f8 INFO: 172.20.0.1:54408 - "POST /node/69f2d852e2fa3d7693ba87f8 HTTP/1.0" 202 Accepted 2026-05-02 04:19:41,982 - lava_callback - INFO - Starting async job submit for node 69f2d852e2fa3d7693ba87f8 2026-05-02 04:19:41,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:19:41,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d852e2fa3d7693ba87f8 HTTP/1.1" 200 None 2026-05-02 04:19:41,997 - lava_callback - DEBUG - Processing node 69f2d852e2fa3d7693ba87f8: baseline-arm64-baylibre 2026-05-02 04:19:41,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d852e2fa3d7693ba87f8 2026-05-02 04:19:41,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:19:42,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d852e2fa3d7693ba87f8 HTTP/1.1" 200 None 2026-05-02 04:19:42,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:19:42,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:19:42,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:19:42,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d852e2fa3d7693ba87f8 HTTP/1.1" 200 None 2026-05-02 04:19:42,052 - lava_callback - ERROR - Error processing callback for node 69f2d852e2fa3d7693ba87f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48486 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:21:02,666 - lava_callback - INFO - Received callback for node 69f2d8a4e2fa3d7693ba881b INFO: 172.20.0.1:43882 - "POST /node/69f2d8a4e2fa3d7693ba881b HTTP/1.0" 202 Accepted 2026-05-02 04:21:02,672 - lava_callback - INFO - Starting async job submit for node 69f2d8a4e2fa3d7693ba881b 2026-05-02 04:21:02,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:21:02,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d8a4e2fa3d7693ba881b HTTP/1.1" 200 None 2026-05-02 04:21:02,690 - lava_callback - DEBUG - Processing node 69f2d8a4e2fa3d7693ba881b: baseline-arm64-clabbe 2026-05-02 04:21:02,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d8a4e2fa3d7693ba881b 2026-05-02 04:21:02,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:21:02,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d8a4e2fa3d7693ba881b HTTP/1.1" 200 None 2026-05-02 04:21:02,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:21:02,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:21:02,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:21:02,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d8a4e2fa3d7693ba881b HTTP/1.1" 200 None 2026-05-02 04:21:02,761 - lava_callback - ERROR - Error processing callback for node 69f2d8a4e2fa3d7693ba881b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38168 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:22:22,707 - lava_callback - INFO - Received callback for node 69f2d8f0e2fa3d7693ba884d INFO: 172.20.0.1:46560 - "POST /node/69f2d8f0e2fa3d7693ba884d HTTP/1.0" 202 Accepted 2026-05-02 04:22:22,712 - lava_callback - INFO - Starting async job submit for node 69f2d8f0e2fa3d7693ba884d 2026-05-02 04:22:22,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:22:22,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d8f0e2fa3d7693ba884d HTTP/1.1" 200 None 2026-05-02 04:22:22,728 - lava_callback - DEBUG - Processing node 69f2d8f0e2fa3d7693ba884d: baseline-arm64-clabbe 2026-05-02 04:22:22,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d8f0e2fa3d7693ba884d 2026-05-02 04:22:22,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:22:22,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d8f0e2fa3d7693ba884d HTTP/1.1" 200 None 2026-05-02 04:22:22,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:22:22,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 04:22:22,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:22:22,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d8f0e2fa3d7693ba884d HTTP/1.1" 200 None 2026-05-02 04:22:22,772 - lava_callback - ERROR - Error processing callback for node 69f2d8f0e2fa3d7693ba884d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47026 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:23:22,148 - lava_callback - INFO - Received callback for node 69f2d935e2fa3d7693ba886c INFO: 172.20.0.1:42174 - "POST /node/69f2d935e2fa3d7693ba886c HTTP/1.0" 202 Accepted 2026-05-02 04:23:22,155 - lava_callback - INFO - Starting async job submit for node 69f2d935e2fa3d7693ba886c 2026-05-02 04:23:22,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:23:22,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d935e2fa3d7693ba886c HTTP/1.1" 200 None 2026-05-02 04:23:22,173 - lava_callback - DEBUG - Processing node 69f2d935e2fa3d7693ba886c: baseline-arm64-baylibre 2026-05-02 04:23:22,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d935e2fa3d7693ba886c 2026-05-02 04:23:22,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:23:22,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d935e2fa3d7693ba886c HTTP/1.1" 200 None 2026-05-02 04:23:22,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:23:22,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:23:22,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:23:22,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d935e2fa3d7693ba886c HTTP/1.1" 200 None 2026-05-02 04:23:22,237 - lava_callback - ERROR - Error processing callback for node 69f2d935e2fa3d7693ba886c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58890 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:24:42,224 - lava_callback - INFO - Received callback for node 69f2d97de2fa3d7693ba889b INFO: 172.20.0.1:43448 - "POST /node/69f2d97de2fa3d7693ba889b HTTP/1.0" 202 Accepted 2026-05-02 04:24:42,229 - lava_callback - INFO - Starting async job submit for node 69f2d97de2fa3d7693ba889b 2026-05-02 04:24:42,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:24:42,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d97de2fa3d7693ba889b HTTP/1.1" 200 None 2026-05-02 04:24:42,247 - lava_callback - DEBUG - Processing node 69f2d97de2fa3d7693ba889b: baseline-arm64-baylibre 2026-05-02 04:24:42,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d97de2fa3d7693ba889b 2026-05-02 04:24:42,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:24:42,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d97de2fa3d7693ba889b HTTP/1.1" 200 None 2026-05-02 04:24:42,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:24:42,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:24:42,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:24:42,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d97de2fa3d7693ba889b HTTP/1.1" 200 None 2026-05-02 04:24:42,315 - lava_callback - ERROR - Error processing callback for node 69f2d97de2fa3d7693ba889b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54228 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:25:42,229 - lava_callback - INFO - Received callback for node 69f2d9bce2fa3d7693ba88be INFO: 172.20.0.1:45694 - "POST /node/69f2d9bce2fa3d7693ba88be HTTP/1.0" 202 Accepted 2026-05-02 04:25:42,234 - lava_callback - INFO - Starting async job submit for node 69f2d9bce2fa3d7693ba88be 2026-05-02 04:25:42,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:25:42,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d9bce2fa3d7693ba88be HTTP/1.1" 200 None 2026-05-02 04:25:42,251 - lava_callback - DEBUG - Processing node 69f2d9bce2fa3d7693ba88be: baseline-arm-baylibre 2026-05-02 04:25:42,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d9bce2fa3d7693ba88be 2026-05-02 04:25:42,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:25:42,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d9bce2fa3d7693ba88be HTTP/1.1" 200 None 2026-05-02 04:25:42,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:25:42,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:25:42,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:25:42,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d9bce2fa3d7693ba88be HTTP/1.1" 200 None 2026-05-02 04:25:42,315 - lava_callback - ERROR - Error processing callback for node 69f2d9bce2fa3d7693ba88be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51858 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:26:42,458 - lava_callback - INFO - Received callback for node 69f2d9ede2fa3d7693ba88ce INFO: 172.20.0.1:45360 - "POST /node/69f2d9ede2fa3d7693ba88ce HTTP/1.0" 202 Accepted 2026-05-02 04:26:42,464 - lava_callback - INFO - Starting async job submit for node 69f2d9ede2fa3d7693ba88ce 2026-05-02 04:26:42,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:26:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d9ede2fa3d7693ba88ce HTTP/1.1" 200 None 2026-05-02 04:26:42,482 - lava_callback - DEBUG - Processing node 69f2d9ede2fa3d7693ba88ce: baseline-arm64-baylibre 2026-05-02 04:26:42,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2d9ede2fa3d7693ba88ce 2026-05-02 04:26:42,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:26:42,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2d9ede2fa3d7693ba88ce HTTP/1.1" 200 None 2026-05-02 04:26:42,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:26:42,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:26:42,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:26:42,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2d9ede2fa3d7693ba88ce HTTP/1.1" 200 None 2026-05-02 04:26:42,538 - lava_callback - ERROR - Error processing callback for node 69f2d9ede2fa3d7693ba88ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50234 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:27:46,367 - lava_callback - INFO - Received callback for node 69f2da2de2fa3d7693ba88ef INFO: 172.20.0.1:54286 - "POST /node/69f2da2de2fa3d7693ba88ef HTTP/1.0" 202 Accepted 2026-05-02 04:27:46,372 - lava_callback - INFO - Starting async job submit for node 69f2da2de2fa3d7693ba88ef 2026-05-02 04:27:46,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:27:46,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2da2de2fa3d7693ba88ef HTTP/1.1" 200 None 2026-05-02 04:27:46,388 - lava_callback - DEBUG - Processing node 69f2da2de2fa3d7693ba88ef: baseline-arm-baylibre 2026-05-02 04:27:46,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2da2de2fa3d7693ba88ef 2026-05-02 04:27:46,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:27:46,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2da2de2fa3d7693ba88ef HTTP/1.1" 200 None 2026-05-02 04:27:46,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:27:46,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:27:46,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:27:46,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2da2de2fa3d7693ba88ef HTTP/1.1" 200 None 2026-05-02 04:27:46,437 - lava_callback - ERROR - Error processing callback for node 69f2da2de2fa3d7693ba88ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:28:02,421 - lava_callback - INFO - Received callback for node 69f2da50e2fa3d7693ba8903 INFO: 172.20.0.1:38442 - "POST /node/69f2da50e2fa3d7693ba8903 HTTP/1.0" 202 Accepted 2026-05-02 04:28:02,427 - lava_callback - INFO - Starting async job submit for node 69f2da50e2fa3d7693ba8903 2026-05-02 04:28:02,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:02,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2da50e2fa3d7693ba8903 HTTP/1.1" 200 None 2026-05-02 04:28:02,445 - lava_callback - DEBUG - Processing node 69f2da50e2fa3d7693ba8903: baseline-arm-baylibre 2026-05-02 04:28:02,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2da50e2fa3d7693ba8903 2026-05-02 04:28:02,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:02,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2da50e2fa3d7693ba8903 HTTP/1.1" 200 None 2026-05-02 04:28:02,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:02,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:28:02,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:02,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2da50e2fa3d7693ba8903 HTTP/1.1" 200 None 2026-05-02 04:28:02,499 - lava_callback - ERROR - Error processing callback for node 69f2da50e2fa3d7693ba8903: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38140 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:28:42,423 - lava_callback - INFO - Received callback for node 69f2da74e2fa3d7693ba890e INFO: 172.20.0.1:39854 - "POST /node/69f2da74e2fa3d7693ba890e HTTP/1.0" 202 Accepted 2026-05-02 04:28:42,429 - lava_callback - INFO - Starting async job submit for node 69f2da74e2fa3d7693ba890e 2026-05-02 04:28:42,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:42,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2da74e2fa3d7693ba890e HTTP/1.1" 200 None 2026-05-02 04:28:42,454 - lava_callback - DEBUG - Processing node 69f2da74e2fa3d7693ba890e: baseline-arm-baylibre 2026-05-02 04:28:42,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2da74e2fa3d7693ba890e 2026-05-02 04:28:42,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:42,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2da74e2fa3d7693ba890e HTTP/1.1" 200 None 2026-05-02 04:28:42,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:42,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:28:42,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:28:42,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2da74e2fa3d7693ba890e HTTP/1.1" 200 None 2026-05-02 04:28:42,510 - lava_callback - ERROR - Error processing callback for node 69f2da74e2fa3d7693ba890e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:29:42,501 - lava_callback - INFO - Received callback for node 69f2daa4e2fa3d7693ba891f INFO: 172.20.0.1:38878 - "POST /node/69f2daa4e2fa3d7693ba891f HTTP/1.0" 202 Accepted 2026-05-02 04:29:42,507 - lava_callback - INFO - Starting async job submit for node 69f2daa4e2fa3d7693ba891f 2026-05-02 04:29:42,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:29:42,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2daa4e2fa3d7693ba891f HTTP/1.1" 200 None 2026-05-02 04:29:42,522 - lava_callback - DEBUG - Processing node 69f2daa4e2fa3d7693ba891f: baseline-arm64-baylibre 2026-05-02 04:29:42,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2daa4e2fa3d7693ba891f 2026-05-02 04:29:42,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:29:42,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2daa4e2fa3d7693ba891f HTTP/1.1" 200 None 2026-05-02 04:29:42,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:29:42,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:29:42,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:29:42,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2daa4e2fa3d7693ba891f HTTP/1.1" 200 None 2026-05-02 04:29:42,569 - lava_callback - ERROR - Error processing callback for node 69f2daa4e2fa3d7693ba891f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33218 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:30:42,517 - lava_callback - INFO - Received callback for node 69f2dae3e2fa3d7693ba8948 INFO: 172.20.0.1:59528 - "POST /node/69f2dae3e2fa3d7693ba8948 HTTP/1.0" 202 Accepted 2026-05-02 04:30:42,522 - lava_callback - INFO - Starting async job submit for node 69f2dae3e2fa3d7693ba8948 2026-05-02 04:30:42,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:30:42,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dae3e2fa3d7693ba8948 HTTP/1.1" 200 None 2026-05-02 04:30:42,544 - lava_callback - DEBUG - Processing node 69f2dae3e2fa3d7693ba8948: baseline-arm-baylibre 2026-05-02 04:30:42,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dae3e2fa3d7693ba8948 2026-05-02 04:30:42,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:30:42,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dae3e2fa3d7693ba8948 HTTP/1.1" 200 None 2026-05-02 04:30:42,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:30:42,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:30:42,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:30:42,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dae3e2fa3d7693ba8948 HTTP/1.1" 200 None 2026-05-02 04:30:42,589 - lava_callback - ERROR - Error processing callback for node 69f2dae3e2fa3d7693ba8948: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:31:43,067 - lava_callback - INFO - Received callback for node 69f2db1ae2fa3d7693ba8954 INFO: 172.20.0.1:48388 - "POST /node/69f2db1ae2fa3d7693ba8954 HTTP/1.0" 202 Accepted 2026-05-02 04:31:43,072 - lava_callback - INFO - Starting async job submit for node 69f2db1ae2fa3d7693ba8954 2026-05-02 04:31:43,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:31:43,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2db1ae2fa3d7693ba8954 HTTP/1.1" 200 None 2026-05-02 04:31:43,087 - lava_callback - DEBUG - Processing node 69f2db1ae2fa3d7693ba8954: baseline-arm64-clabbe 2026-05-02 04:31:43,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2db1ae2fa3d7693ba8954 2026-05-02 04:31:43,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:31:43,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2db1ae2fa3d7693ba8954 HTTP/1.1" 200 None 2026-05-02 04:31:43,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:31:43,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:31:43,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:31:43,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2db1ae2fa3d7693ba8954 HTTP/1.1" 200 None 2026-05-02 04:31:43,134 - lava_callback - ERROR - Error processing callback for node 69f2db1ae2fa3d7693ba8954: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43590 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:32:42,636 - lava_callback - INFO - Received callback for node 69f2db59e2fa3d7693ba8975 INFO: 172.20.0.1:35354 - "POST /node/69f2db59e2fa3d7693ba8975 HTTP/1.0" 202 Accepted 2026-05-02 04:32:42,641 - lava_callback - INFO - Starting async job submit for node 69f2db59e2fa3d7693ba8975 2026-05-02 04:32:42,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:32:42,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2db59e2fa3d7693ba8975 HTTP/1.1" 200 None 2026-05-02 04:32:42,656 - lava_callback - DEBUG - Processing node 69f2db59e2fa3d7693ba8975: baseline-arm-baylibre 2026-05-02 04:32:42,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2db59e2fa3d7693ba8975 2026-05-02 04:32:42,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:32:42,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2db59e2fa3d7693ba8975 HTTP/1.1" 200 None 2026-05-02 04:32:42,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:32:42,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:32:42,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:32:42,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2db59e2fa3d7693ba8975 HTTP/1.1" 200 None 2026-05-02 04:32:42,701 - lava_callback - ERROR - Error processing callback for node 69f2db59e2fa3d7693ba8975: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:33:23,166 - lava_callback - INFO - Received callback for node 69f2db8ee2fa3d7693ba898d INFO: 172.20.0.1:48388 - "POST /node/69f2db8ee2fa3d7693ba898d HTTP/1.0" 202 Accepted 2026-05-02 04:33:23,172 - lava_callback - INFO - Starting async job submit for node 69f2db8ee2fa3d7693ba898d 2026-05-02 04:33:23,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:33:23,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2db8ee2fa3d7693ba898d HTTP/1.1" 200 None 2026-05-02 04:33:23,190 - lava_callback - DEBUG - Processing node 69f2db8ee2fa3d7693ba898d: baseline-arm64-clabbe 2026-05-02 04:33:23,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2db8ee2fa3d7693ba898d 2026-05-02 04:33:23,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:33:23,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2db8ee2fa3d7693ba898d HTTP/1.1" 200 None 2026-05-02 04:33:23,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:33:23,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:33:23,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:33:23,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2db8ee2fa3d7693ba898d HTTP/1.1" 200 None 2026-05-02 04:33:23,249 - lava_callback - ERROR - Error processing callback for node 69f2db8ee2fa3d7693ba898d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42832 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:34:43,187 - lava_callback - INFO - Received callback for node 69f2dbd7e2fa3d7693ba89b5 INFO: 172.20.0.1:36320 - "POST /node/69f2dbd7e2fa3d7693ba89b5 HTTP/1.0" 202 Accepted 2026-05-02 04:34:43,193 - lava_callback - INFO - Starting async job submit for node 69f2dbd7e2fa3d7693ba89b5 2026-05-02 04:34:43,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:34:43,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dbd7e2fa3d7693ba89b5 HTTP/1.1" 200 None 2026-05-02 04:34:43,208 - lava_callback - DEBUG - Processing node 69f2dbd7e2fa3d7693ba89b5: baseline-arm-clabbe 2026-05-02 04:34:43,208 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dbd7e2fa3d7693ba89b5 2026-05-02 04:34:43,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:34:43,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dbd7e2fa3d7693ba89b5 HTTP/1.1" 200 None 2026-05-02 04:34:43,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:34:43,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:34:43,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:34:43,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dbd7e2fa3d7693ba89b5 HTTP/1.1" 200 None 2026-05-02 04:34:43,264 - lava_callback - ERROR - Error processing callback for node 69f2dbd7e2fa3d7693ba89b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:35:00,154 - lava_callback - INFO - Received callback for node 69f57ee5ed4ed40b0224202e INFO: 172.20.0.1:53862 - "POST /node/69f57ee5ed4ed40b0224202e HTTP/1.0" 202 Accepted 2026-05-02 04:35:00,160 - lava_callback - INFO - Starting async job submit for node 69f57ee5ed4ed40b0224202e 2026-05-02 04:35:00,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:00,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f57ee5ed4ed40b0224202e HTTP/1.1" 200 None 2026-05-02 04:35:00,184 - lava_callback - DEBUG - Processing node 69f57ee5ed4ed40b0224202e: baseline-nfs-arm64 2026-05-02 04:35:00,185 - lava_callback - WARNING - Failed to upload log 2026-05-02 04:35:00,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:35:00,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:35:00,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f57ee5ed4ed40b0224202e/lava-logs.yaml 2026-05-02 04:35:00,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:35:00,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:35:00,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f57ee5ed4ed40b0224202e/lava_callback.json.gz 2026-05-02 04:35:00,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:00,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f57ee5ed4ed40b0224202e HTTP/1.1" 200 None 2026-05-02 04:35:00,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:00,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 04:35:00,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:00,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f57ee5ed4ed40b0224202e HTTP/1.1" 200 None 2026-05-02 04:35:00,453 - lava_callback - INFO - Completed processing callback for node 69f57ee5ed4ed40b0224202e INFO: 172.20.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52330 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:35:23,133 - lava_callback - INFO - Received callback for node 69f2dbfce2fa3d7693ba89c1 INFO: 172.20.0.1:51644 - "POST /node/69f2dbfce2fa3d7693ba89c1 HTTP/1.0" 202 Accepted 2026-05-02 04:35:23,138 - lava_callback - INFO - Starting async job submit for node 69f2dbfce2fa3d7693ba89c1 2026-05-02 04:35:23,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:23,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dbfce2fa3d7693ba89c1 HTTP/1.1" 200 None 2026-05-02 04:35:23,153 - lava_callback - DEBUG - Processing node 69f2dbfce2fa3d7693ba89c1: baseline-arm-clabbe 2026-05-02 04:35:23,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dbfce2fa3d7693ba89c1 2026-05-02 04:35:23,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:23,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dbfce2fa3d7693ba89c1 HTTP/1.1" 200 None 2026-05-02 04:35:23,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:23,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:35:23,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:23,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dbfce2fa3d7693ba89c1 HTTP/1.1" 200 None 2026-05-02 04:35:23,224 - lava_callback - ERROR - Error processing callback for node 69f2dbfce2fa3d7693ba89c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-02 04:35:23,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:35:23,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35250 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:36:03,251 - lava_callback - INFO - Received callback for node 69f2dc2fe2fa3d7693ba89d8 INFO: 172.20.0.1:46062 - "POST /node/69f2dc2fe2fa3d7693ba89d8 HTTP/1.0" 202 Accepted 2026-05-02 04:36:03,259 - lava_callback - INFO - Starting async job submit for node 69f2dc2fe2fa3d7693ba89d8 2026-05-02 04:36:03,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:36:03,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dc2fe2fa3d7693ba89d8 HTTP/1.1" 200 None 2026-05-02 04:36:03,275 - lava_callback - DEBUG - Processing node 69f2dc2fe2fa3d7693ba89d8: baseline-arm64-clabbe 2026-05-02 04:36:03,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dc2fe2fa3d7693ba89d8 2026-05-02 04:36:03,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:36:03,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dc2fe2fa3d7693ba89d8 HTTP/1.1" 200 None 2026-05-02 04:36:03,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:36:03,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:36:03,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:36:03,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dc2fe2fa3d7693ba89d8 HTTP/1.1" 200 None 2026-05-02 04:36:03,339 - lava_callback - ERROR - Error processing callback for node 69f2dc2fe2fa3d7693ba89d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39102 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:37:29,177 - lava_callback - INFO - Received callback for node 69f57aeded4ed40b02241eb3 INFO: 172.20.0.1:39112 - "POST /node/69f57aeded4ed40b02241eb3 HTTP/1.0" 202 Accepted 2026-05-02 04:37:29,192 - lava_callback - INFO - Starting async job submit for node 69f57aeded4ed40b02241eb3 2026-05-02 04:37:29,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:37:29,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f57aeded4ed40b02241eb3 HTTP/1.1" 200 None 2026-05-02 04:37:29,251 - lava_callback - DEBUG - Processing node 69f57aeded4ed40b02241eb3: baseline-nfs-arm64 2026-05-02 04:37:30,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:37:30,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:37:30,469 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f57aeded4ed40b02241eb3/log.txt.gz 2026-05-02 04:37:31,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:37:31,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:37:31,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f57aeded4ed40b02241eb3/lava-logs.yaml 2026-05-02 04:37:31,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:37:31,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:37:31,578 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f57aeded4ed40b02241eb3/lava_callback.json.gz DEBUG: Setup failed for 69f57aeded4ed40b02241eb3. Transitting job node result: incomplete -> fail 2026-05-02 04:37:31,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:37:31,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f57aeded4ed40b02241eb3 HTTP/1.1" 200 None 2026-05-02 04:37:31,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:37:31,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 04:37:31,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:37:31,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f57aeded4ed40b02241eb3 HTTP/1.1" 200 None 2026-05-02 04:37:31,789 - lava_callback - INFO - Completed processing callback for node 69f57aeded4ed40b02241eb3 INFO: 172.20.0.1:40082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51598 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:37:53,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:37:53,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51384 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:38:43,374 - lava_callback - INFO - Received callback for node 69f2dcc2e2fa3d7693ba8a16 INFO: 172.20.0.1:49152 - "POST /node/69f2dcc2e2fa3d7693ba8a16 HTTP/1.0" 202 Accepted 2026-05-02 04:38:43,380 - lava_callback - INFO - Starting async job submit for node 69f2dcc2e2fa3d7693ba8a16 2026-05-02 04:38:43,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:38:43,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dcc2e2fa3d7693ba8a16 HTTP/1.1" 200 None 2026-05-02 04:38:43,395 - lava_callback - DEBUG - Processing node 69f2dcc2e2fa3d7693ba8a16: baseline-arm-clabbe 2026-05-02 04:38:43,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dcc2e2fa3d7693ba8a16 2026-05-02 04:38:43,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:38:43,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dcc2e2fa3d7693ba8a16 HTTP/1.1" 200 None 2026-05-02 04:38:43,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:38:43,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:38:43,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:38:43,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dcc2e2fa3d7693ba8a16 HTTP/1.1" 200 None 2026-05-02 04:38:43,444 - lava_callback - ERROR - Error processing callback for node 69f2dcc2e2fa3d7693ba8a16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41248 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:39:23,280 - lava_callback - INFO - Received callback for node 69f2dce7e2fa3d7693ba8a2f INFO: 172.20.0.1:53676 - "POST /node/69f2dce7e2fa3d7693ba8a2f HTTP/1.0" 202 Accepted 2026-05-02 04:39:23,286 - lava_callback - INFO - Starting async job submit for node 69f2dce7e2fa3d7693ba8a2f 2026-05-02 04:39:23,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:39:23,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dce7e2fa3d7693ba8a2f HTTP/1.1" 200 None 2026-05-02 04:39:23,413 - lava_callback - DEBUG - Processing node 69f2dce7e2fa3d7693ba8a2f: baseline-arm-clabbe 2026-05-02 04:39:23,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dce7e2fa3d7693ba8a2f 2026-05-02 04:39:23,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:39:23,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dce7e2fa3d7693ba8a2f HTTP/1.1" 200 None 2026-05-02 04:39:23,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:39:23,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:39:23,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:39:23,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dce7e2fa3d7693ba8a2f HTTP/1.1" 200 None 2026-05-02 04:39:23,545 - lava_callback - ERROR - Error processing callback for node 69f2dce7e2fa3d7693ba8a2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35558 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:40:03,493 - lava_callback - INFO - Received callback for node 69f2dd19e2fa3d7693ba8a3b INFO: 172.20.0.1:38458 - "POST /node/69f2dd19e2fa3d7693ba8a3b HTTP/1.0" 202 Accepted 2026-05-02 04:40:03,498 - lava_callback - INFO - Starting async job submit for node 69f2dd19e2fa3d7693ba8a3b 2026-05-02 04:40:03,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:40:03,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dd19e2fa3d7693ba8a3b HTTP/1.1" 200 None 2026-05-02 04:40:03,513 - lava_callback - DEBUG - Processing node 69f2dd19e2fa3d7693ba8a3b: baseline-arm64-clabbe 2026-05-02 04:40:03,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dd19e2fa3d7693ba8a3b 2026-05-02 04:40:03,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:40:03,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dd19e2fa3d7693ba8a3b HTTP/1.1" 200 None 2026-05-02 04:40:03,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:40:03,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:40:03,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:40:03,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dd19e2fa3d7693ba8a3b HTTP/1.1" 200 None 2026-05-02 04:40:03,557 - lava_callback - ERROR - Error processing callback for node 69f2dd19e2fa3d7693ba8a3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:41:03,405 - lava_callback - INFO - Received callback for node 69f2dd57e2fa3d7693ba8a5f INFO: 172.20.0.1:41390 - "POST /node/69f2dd57e2fa3d7693ba8a5f HTTP/1.0" 202 Accepted 2026-05-02 04:41:03,410 - lava_callback - INFO - Starting async job submit for node 69f2dd57e2fa3d7693ba8a5f 2026-05-02 04:41:03,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:03,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dd57e2fa3d7693ba8a5f HTTP/1.1" 200 None 2026-05-02 04:41:03,426 - lava_callback - DEBUG - Processing node 69f2dd57e2fa3d7693ba8a5f: baseline-arm-clabbe 2026-05-02 04:41:03,426 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dd57e2fa3d7693ba8a5f 2026-05-02 04:41:03,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:03,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dd57e2fa3d7693ba8a5f HTTP/1.1" 200 None 2026-05-02 04:41:03,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:03,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:41:03,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:03,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dd57e2fa3d7693ba8a5f HTTP/1.1" 200 None 2026-05-02 04:41:03,480 - lava_callback - ERROR - Error processing callback for node 69f2dd57e2fa3d7693ba8a5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58288 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:41:40,256 - lava_callback - INFO - Received callback for node 69f5804ded4ed40b022420b9 INFO: 172.20.0.1:34590 - "POST /node/69f5804ded4ed40b022420b9 HTTP/1.0" 202 Accepted 2026-05-02 04:41:40,263 - lava_callback - INFO - Starting async job submit for node 69f5804ded4ed40b022420b9 2026-05-02 04:41:40,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:40,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5804ded4ed40b022420b9 HTTP/1.1" 200 None 2026-05-02 04:41:40,316 - lava_callback - DEBUG - Processing node 69f5804ded4ed40b022420b9: baseline-arm64 2026-05-02 04:41:40,477 - lava_callback - WARNING - Failed to upload log 2026-05-02 04:41:40,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:41:40,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:41:40,732 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f5804ded4ed40b022420b9/lava-logs.yaml 2026-05-02 04:41:40,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:41:40,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:41:40,824 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f5804ded4ed40b022420b9/lava_callback.json.gz 2026-05-02 04:41:40,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:40,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5804ded4ed40b022420b9 HTTP/1.1" 200 None 2026-05-02 04:41:40,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:40,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 04:41:40,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:40,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f5804ded4ed40b022420b9 HTTP/1.1" 200 None 2026-05-02 04:41:40,958 - lava_callback - INFO - Completed processing callback for node 69f5804ded4ed40b022420b9 INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:41:53,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:41:53,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47462 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:43:00,271 - lava_callback - INFO - Received callback for node 69f58098ed4ed40b022420de INFO: 172.20.0.1:51336 - "POST /node/69f58098ed4ed40b022420de HTTP/1.0" 202 Accepted 2026-05-02 04:43:00,277 - lava_callback - INFO - Starting async job submit for node 69f58098ed4ed40b022420de 2026-05-02 04:43:00,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:00,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58098ed4ed40b022420de HTTP/1.1" 200 None 2026-05-02 04:43:00,320 - lava_callback - DEBUG - Processing node 69f58098ed4ed40b022420de: baseline-arm64 2026-05-02 04:43:00,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:43:00,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:43:00,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58098ed4ed40b022420de/log.txt.gz 2026-05-02 04:43:00,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:43:00,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:43:00,763 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58098ed4ed40b022420de/lava-logs.yaml 2026-05-02 04:43:00,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:43:00,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:43:00,853 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58098ed4ed40b022420de/lava_callback.json.gz 2026-05-02 04:43:00,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:00,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58098ed4ed40b022420de HTTP/1.1" 200 None 2026-05-02 04:43:00,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:00,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 04:43:00,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:00,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f58098ed4ed40b022420de HTTP/1.1" 200 None 2026-05-02 04:43:00,993 - lava_callback - INFO - Completed processing callback for node 69f58098ed4ed40b022420de 2026-05-02 04:43:03,548 - lava_callback - INFO - Received callback for node 69f2ddc6e2fa3d7693ba8a96 INFO: 172.20.0.1:51346 - "POST /node/69f2ddc6e2fa3d7693ba8a96 HTTP/1.0" 202 Accepted 2026-05-02 04:43:03,554 - lava_callback - INFO - Starting async job submit for node 69f2ddc6e2fa3d7693ba8a96 2026-05-02 04:43:03,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:03,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ddc6e2fa3d7693ba8a96 HTTP/1.1" 200 None 2026-05-02 04:43:03,570 - lava_callback - DEBUG - Processing node 69f2ddc6e2fa3d7693ba8a96: baseline-arm-clabbe 2026-05-02 04:43:03,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ddc6e2fa3d7693ba8a96 2026-05-02 04:43:03,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:03,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ddc6e2fa3d7693ba8a96 HTTP/1.1" 200 None 2026-05-02 04:43:03,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:03,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:43:03,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:03,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ddc6e2fa3d7693ba8a96 HTTP/1.1" 200 None 2026-05-02 04:43:03,616 - lava_callback - ERROR - Error processing callback for node 69f2ddc6e2fa3d7693ba8a96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56814 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:43:23,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:23,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56676 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:43:43,543 - lava_callback - INFO - Received callback for node 69f2ddebe2fa3d7693ba8aa8 INFO: 172.20.0.1:44196 - "POST /node/69f2ddebe2fa3d7693ba8aa8 HTTP/1.0" 202 Accepted 2026-05-02 04:43:43,549 - lava_callback - INFO - Starting async job submit for node 69f2ddebe2fa3d7693ba8aa8 2026-05-02 04:43:43,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:43,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ddebe2fa3d7693ba8aa8 HTTP/1.1" 200 None 2026-05-02 04:43:43,564 - lava_callback - DEBUG - Processing node 69f2ddebe2fa3d7693ba8aa8: baseline-arm-clabbe 2026-05-02 04:43:43,564 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ddebe2fa3d7693ba8aa8 2026-05-02 04:43:43,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:43,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ddebe2fa3d7693ba8aa8 HTTP/1.1" 200 None 2026-05-02 04:43:43,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:43,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:43:43,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:43:43,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ddebe2fa3d7693ba8aa8 HTTP/1.1" 200 None 2026-05-02 04:43:43,610 - lava_callback - ERROR - Error processing callback for node 69f2ddebe2fa3d7693ba8aa8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41010 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:45:23,674 - lava_callback - INFO - Received callback for node 69f2de5ae2fa3d7693ba8adb INFO: 172.20.0.1:56590 - "POST /node/69f2de5ae2fa3d7693ba8adb HTTP/1.0" 202 Accepted 2026-05-02 04:45:23,680 - lava_callback - INFO - Starting async job submit for node 69f2de5ae2fa3d7693ba8adb 2026-05-02 04:45:23,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:45:23,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2de5ae2fa3d7693ba8adb HTTP/1.1" 200 None 2026-05-02 04:45:23,695 - lava_callback - DEBUG - Processing node 69f2de5ae2fa3d7693ba8adb: baseline-arm-clabbe 2026-05-02 04:45:23,695 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2de5ae2fa3d7693ba8adb 2026-05-02 04:45:23,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:45:23,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2de5ae2fa3d7693ba8adb HTTP/1.1" 200 None 2026-05-02 04:45:23,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:45:23,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:45:23,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:45:23,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2de5ae2fa3d7693ba8adb HTTP/1.1" 200 None 2026-05-02 04:45:23,746 - lava_callback - ERROR - Error processing callback for node 69f2de5ae2fa3d7693ba8adb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:46:03,623 - lava_callback - INFO - Received callback for node 69f2de7ee2fa3d7693ba8af1 INFO: 172.20.0.1:55286 - "POST /node/69f2de7ee2fa3d7693ba8af1 HTTP/1.0" 202 Accepted 2026-05-02 04:46:03,628 - lava_callback - INFO - Starting async job submit for node 69f2de7ee2fa3d7693ba8af1 2026-05-02 04:46:03,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:46:03,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2de7ee2fa3d7693ba8af1 HTTP/1.1" 200 None 2026-05-02 04:46:03,643 - lava_callback - DEBUG - Processing node 69f2de7ee2fa3d7693ba8af1: baseline-arm-clabbe 2026-05-02 04:46:03,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2de7ee2fa3d7693ba8af1 2026-05-02 04:46:03,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:46:03,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2de7ee2fa3d7693ba8af1 HTTP/1.1" 200 None 2026-05-02 04:46:03,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:46:03,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 04:46:03,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:46:03,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2de7ee2fa3d7693ba8af1 HTTP/1.1" 200 None 2026-05-02 04:46:03,695 - lava_callback - ERROR - Error processing callback for node 69f2de7ee2fa3d7693ba8af1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36790 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:49:03,652 - lava_callback - INFO - Received callback for node 69f2df33e2fa3d7693ba8b4d INFO: 172.20.0.1:53134 - "POST /node/69f2df33e2fa3d7693ba8b4d HTTP/1.0" 202 Accepted 2026-05-02 04:49:03,658 - lava_callback - INFO - Starting async job submit for node 69f2df33e2fa3d7693ba8b4d 2026-05-02 04:49:03,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:03,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2df33e2fa3d7693ba8b4d HTTP/1.1" 200 None 2026-05-02 04:49:03,679 - lava_callback - DEBUG - Processing node 69f2df33e2fa3d7693ba8b4d: baseline-arm-baylibre 2026-05-02 04:49:03,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2df33e2fa3d7693ba8b4d 2026-05-02 04:49:03,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:03,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2df33e2fa3d7693ba8b4d HTTP/1.1" 200 None 2026-05-02 04:49:03,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:03,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:49:03,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:03,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2df33e2fa3d7693ba8b4d HTTP/1.1" 200 None 2026-05-02 04:49:03,728 - lava_callback - ERROR - Error processing callback for node 69f2df33e2fa3d7693ba8b4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:49:43,621 - lava_callback - INFO - Received callback for node 69f2df58e2fa3d7693ba8b60 INFO: 172.20.0.1:37446 - "POST /node/69f2df58e2fa3d7693ba8b60 HTTP/1.0" 202 Accepted 2026-05-02 04:49:43,627 - lava_callback - INFO - Starting async job submit for node 69f2df58e2fa3d7693ba8b60 2026-05-02 04:49:43,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:43,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2df58e2fa3d7693ba8b60 HTTP/1.1" 200 None 2026-05-02 04:49:43,641 - lava_callback - DEBUG - Processing node 69f2df58e2fa3d7693ba8b60: baseline-arm-baylibre 2026-05-02 04:49:43,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2df58e2fa3d7693ba8b60 2026-05-02 04:49:43,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:43,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2df58e2fa3d7693ba8b60 HTTP/1.1" 200 None 2026-05-02 04:49:43,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:43,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:49:43,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:49:43,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2df58e2fa3d7693ba8b60 HTTP/1.1" 200 None 2026-05-02 04:49:43,693 - lava_callback - ERROR - Error processing callback for node 69f2df58e2fa3d7693ba8b60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46674 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:50:23,858 - lava_callback - INFO - Received callback for node 69f2df7de2fa3d7693ba8b6c INFO: 172.20.0.1:44382 - "POST /node/69f2df7de2fa3d7693ba8b6c HTTP/1.0" 202 Accepted 2026-05-02 04:50:23,863 - lava_callback - INFO - Starting async job submit for node 69f2df7de2fa3d7693ba8b6c 2026-05-02 04:50:23,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:23,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2df7de2fa3d7693ba8b6c HTTP/1.1" 200 None 2026-05-02 04:50:23,879 - lava_callback - DEBUG - Processing node 69f2df7de2fa3d7693ba8b6c: baseline-arm-baylibre 2026-05-02 04:50:23,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2df7de2fa3d7693ba8b6c 2026-05-02 04:50:23,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:23,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2df7de2fa3d7693ba8b6c HTTP/1.1" 200 None 2026-05-02 04:50:23,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:23,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:50:23,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:23,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2df7de2fa3d7693ba8b6c HTTP/1.1" 200 None 2026-05-02 04:50:23,941 - lava_callback - ERROR - Error processing callback for node 69f2df7de2fa3d7693ba8b6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51862 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:50:48,886 - lava_callback - INFO - Received callback for node 69f580e7ed4ed40b02242102 INFO: 172.20.0.1:51876 - "POST /node/69f580e7ed4ed40b02242102 HTTP/1.0" 202 Accepted 2026-05-02 04:50:48,893 - lava_callback - INFO - Starting async job submit for node 69f580e7ed4ed40b02242102 2026-05-02 04:50:48,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:48,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f580e7ed4ed40b02242102 HTTP/1.1" 200 None 2026-05-02 04:50:48,942 - lava_callback - DEBUG - Processing node 69f580e7ed4ed40b02242102: baseline-arm64 2026-05-02 04:50:49,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:50:49,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:50:49,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f580e7ed4ed40b02242102/log.txt.gz 2026-05-02 04:50:49,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:50:49,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:50:49,992 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f580e7ed4ed40b02242102/lava-logs.yaml 2026-05-02 04:50:50,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 04:50:50,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 04:50:50,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f580e7ed4ed40b02242102/lava_callback.json.gz DEBUG: Setup failed for 69f580e7ed4ed40b02242102. Transitting job node result: incomplete -> fail 2026-05-02 04:50:50,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:50,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f580e7ed4ed40b02242102 HTTP/1.1" 200 None 2026-05-02 04:50:50,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:50,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 04:50:50,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:50,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f580e7ed4ed40b02242102 HTTP/1.1" 200 None 2026-05-02 04:50:50,311 - lava_callback - INFO - Completed processing callback for node 69f580e7ed4ed40b02242102 2026-05-02 04:50:53,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:50:53,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45246 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:51:03,825 - lava_callback - INFO - Received callback for node 69f2dfb2e2fa3d7693ba8b88 INFO: 172.20.0.1:55988 - "POST /node/69f2dfb2e2fa3d7693ba8b88 HTTP/1.0" 202 Accepted 2026-05-02 04:51:03,830 - lava_callback - INFO - Starting async job submit for node 69f2dfb2e2fa3d7693ba8b88 2026-05-02 04:51:03,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:51:03,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dfb2e2fa3d7693ba8b88 HTTP/1.1" 200 None 2026-05-02 04:51:03,849 - lava_callback - DEBUG - Processing node 69f2dfb2e2fa3d7693ba8b88: baseline-arm64-clabbe 2026-05-02 04:51:03,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dfb2e2fa3d7693ba8b88 2026-05-02 04:51:03,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:51:03,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dfb2e2fa3d7693ba8b88 HTTP/1.1" 200 None 2026-05-02 04:51:03,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:51:03,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 04:51:03,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:51:03,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dfb2e2fa3d7693ba8b88 HTTP/1.1" 200 None 2026-05-02 04:51:03,893 - lava_callback - ERROR - Error processing callback for node 69f2dfb2e2fa3d7693ba8b88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:52:03,787 - lava_callback - INFO - Received callback for node 69f2dfece2fa3d7693ba8b9e INFO: 172.20.0.1:47544 - "POST /node/69f2dfece2fa3d7693ba8b9e HTTP/1.0" 202 Accepted 2026-05-02 04:52:03,792 - lava_callback - INFO - Starting async job submit for node 69f2dfece2fa3d7693ba8b9e 2026-05-02 04:52:03,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:03,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dfece2fa3d7693ba8b9e HTTP/1.1" 200 None 2026-05-02 04:52:03,809 - lava_callback - DEBUG - Processing node 69f2dfece2fa3d7693ba8b9e: baseline-arm-baylibre 2026-05-02 04:52:03,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2dfece2fa3d7693ba8b9e 2026-05-02 04:52:03,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:03,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2dfece2fa3d7693ba8b9e HTTP/1.1" 200 None 2026-05-02 04:52:03,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:03,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:52:03,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:03,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2dfece2fa3d7693ba8b9e HTTP/1.1" 200 None 2026-05-02 04:52:03,863 - lava_callback - ERROR - Error processing callback for node 69f2dfece2fa3d7693ba8b9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54156 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:52:43,764 - lava_callback - INFO - Received callback for node 69f2e011e2fa3d7693ba8baf INFO: 172.20.0.1:58800 - "POST /node/69f2e011e2fa3d7693ba8baf HTTP/1.0" 202 Accepted 2026-05-02 04:52:43,770 - lava_callback - INFO - Starting async job submit for node 69f2e011e2fa3d7693ba8baf 2026-05-02 04:52:43,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:43,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e011e2fa3d7693ba8baf HTTP/1.1" 200 None 2026-05-02 04:52:43,789 - lava_callback - DEBUG - Processing node 69f2e011e2fa3d7693ba8baf: baseline-arm-baylibre 2026-05-02 04:52:43,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e011e2fa3d7693ba8baf 2026-05-02 04:52:43,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:43,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e011e2fa3d7693ba8baf HTTP/1.1" 200 None 2026-05-02 04:52:43,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:43,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:52:43,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:52:43,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e011e2fa3d7693ba8baf HTTP/1.1" 200 None 2026-05-02 04:52:43,856 - lava_callback - ERROR - Error processing callback for node 69f2e011e2fa3d7693ba8baf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36896 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:53:23,888 - lava_callback - INFO - Received callback for node 69f2e03be2fa3d7693ba8bc0 INFO: 172.20.0.1:58194 - "POST /node/69f2e03be2fa3d7693ba8bc0 HTTP/1.0" 202 Accepted 2026-05-02 04:53:23,894 - lava_callback - INFO - Starting async job submit for node 69f2e03be2fa3d7693ba8bc0 2026-05-02 04:53:23,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:53:23,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e03be2fa3d7693ba8bc0 HTTP/1.1" 200 None 2026-05-02 04:53:23,912 - lava_callback - DEBUG - Processing node 69f2e03be2fa3d7693ba8bc0: baseline-arm64-baylibre 2026-05-02 04:53:23,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e03be2fa3d7693ba8bc0 2026-05-02 04:53:23,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:53:23,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e03be2fa3d7693ba8bc0 HTTP/1.1" 200 None 2026-05-02 04:53:23,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:53:23,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2024713aba9734a49455e HTTP/1.1" 200 None 2026-05-02 04:53:23,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:53:23,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e03be2fa3d7693ba8bc0 HTTP/1.1" 200 None 2026-05-02 04:53:23,963 - lava_callback - ERROR - Error processing callback for node 69f2e03be2fa3d7693ba8bc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45182 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:54:43,985 - lava_callback - INFO - Received callback for node 69f2e07ee2fa3d7693ba8be4 INFO: 172.20.0.1:53276 - "POST /node/69f2e07ee2fa3d7693ba8be4 HTTP/1.0" 202 Accepted 2026-05-02 04:54:43,990 - lava_callback - INFO - Starting async job submit for node 69f2e07ee2fa3d7693ba8be4 2026-05-02 04:54:43,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:54:44,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e07ee2fa3d7693ba8be4 HTTP/1.1" 200 None 2026-05-02 04:54:44,007 - lava_callback - DEBUG - Processing node 69f2e07ee2fa3d7693ba8be4: baseline-arm-baylibre 2026-05-02 04:54:44,007 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e07ee2fa3d7693ba8be4 2026-05-02 04:54:44,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:54:44,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e07ee2fa3d7693ba8be4 HTTP/1.1" 200 None 2026-05-02 04:54:44,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:54:44,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:54:44,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:54:44,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e07ee2fa3d7693ba8be4 HTTP/1.1" 200 None 2026-05-02 04:54:44,052 - lava_callback - ERROR - Error processing callback for node 69f2e07ee2fa3d7693ba8be4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34444 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:55:23,938 - lava_callback - INFO - Received callback for node 69f2e0aae2fa3d7693ba8bfb INFO: 172.20.0.1:59506 - "POST /node/69f2e0aae2fa3d7693ba8bfb HTTP/1.0" 202 Accepted 2026-05-02 04:55:23,946 - lava_callback - INFO - Starting async job submit for node 69f2e0aae2fa3d7693ba8bfb 2026-05-02 04:55:23,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:55:23,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e0aae2fa3d7693ba8bfb HTTP/1.1" 200 None 2026-05-02 04:55:23,968 - lava_callback - DEBUG - Processing node 69f2e0aae2fa3d7693ba8bfb: baseline-arm64-baylibre 2026-05-02 04:55:23,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e0aae2fa3d7693ba8bfb 2026-05-02 04:55:23,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:55:23,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e0aae2fa3d7693ba8bfb HTTP/1.1" 200 None 2026-05-02 04:55:23,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:55:24,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2024713aba9734a49455e HTTP/1.1" 200 None 2026-05-02 04:55:24,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:55:24,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e0aae2fa3d7693ba8bfb HTTP/1.1" 200 None 2026-05-02 04:55:24,027 - lava_callback - ERROR - Error processing callback for node 69f2e0aae2fa3d7693ba8bfb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36044 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:56:24,074 - lava_callback - INFO - Received callback for node 69f2e0f6e2fa3d7693ba8c1a INFO: 172.20.0.1:56912 - "POST /node/69f2e0f6e2fa3d7693ba8c1a HTTP/1.0" 202 Accepted 2026-05-02 04:56:24,079 - lava_callback - INFO - Starting async job submit for node 69f2e0f6e2fa3d7693ba8c1a 2026-05-02 04:56:24,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:56:24,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e0f6e2fa3d7693ba8c1a HTTP/1.1" 200 None 2026-05-02 04:56:24,094 - lava_callback - DEBUG - Processing node 69f2e0f6e2fa3d7693ba8c1a: baseline-arm64-baylibre 2026-05-02 04:56:24,094 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e0f6e2fa3d7693ba8c1a 2026-05-02 04:56:24,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:56:24,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e0f6e2fa3d7693ba8c1a HTTP/1.1" 200 None 2026-05-02 04:56:24,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:56:24,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2024713aba9734a49455e HTTP/1.1" 200 None 2026-05-02 04:56:24,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:56:24,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e0f6e2fa3d7693ba8c1a HTTP/1.1" 200 None 2026-05-02 04:56:24,154 - lava_callback - ERROR - Error processing callback for node 69f2e0f6e2fa3d7693ba8c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39864 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:57:44,064 - lava_callback - INFO - Received callback for node 69f2e13ae2fa3d7693ba8c48 INFO: 172.20.0.1:39828 - "POST /node/69f2e13ae2fa3d7693ba8c48 HTTP/1.0" 202 Accepted 2026-05-02 04:57:44,069 - lava_callback - INFO - Starting async job submit for node 69f2e13ae2fa3d7693ba8c48 2026-05-02 04:57:44,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:57:44,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e13ae2fa3d7693ba8c48 HTTP/1.1" 200 None 2026-05-02 04:57:44,083 - lava_callback - DEBUG - Processing node 69f2e13ae2fa3d7693ba8c48: baseline-arm-clabbe 2026-05-02 04:57:44,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e13ae2fa3d7693ba8c48 2026-05-02 04:57:44,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:57:44,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e13ae2fa3d7693ba8c48 HTTP/1.1" 200 None 2026-05-02 04:57:44,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:57:44,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:57:44,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:57:44,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e13ae2fa3d7693ba8c48 HTTP/1.1" 200 None 2026-05-02 04:57:44,135 - lava_callback - ERROR - Error processing callback for node 69f2e13ae2fa3d7693ba8c48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34214 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:58:24,062 - lava_callback - INFO - Received callback for node 69f2e15fe2fa3d7693ba8c51 INFO: 172.20.0.1:50542 - "POST /node/69f2e15fe2fa3d7693ba8c51 HTTP/1.0" 202 Accepted 2026-05-02 04:58:24,067 - lava_callback - INFO - Starting async job submit for node 69f2e15fe2fa3d7693ba8c51 2026-05-02 04:58:24,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:58:24,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e15fe2fa3d7693ba8c51 HTTP/1.1" 200 None 2026-05-02 04:58:24,082 - lava_callback - DEBUG - Processing node 69f2e15fe2fa3d7693ba8c51: baseline-arm-clabbe 2026-05-02 04:58:24,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e15fe2fa3d7693ba8c51 2026-05-02 04:58:24,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:58:24,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e15fe2fa3d7693ba8c51 HTTP/1.1" 200 None 2026-05-02 04:58:24,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:58:24,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 04:58:24,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:58:24,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e15fe2fa3d7693ba8c51 HTTP/1.1" 200 None 2026-05-02 04:58:24,130 - lava_callback - ERROR - Error processing callback for node 69f2e15fe2fa3d7693ba8c51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40320 - "GET / HTTP/1.0" 200 OK 2026-05-02 04:59:04,191 - lava_callback - INFO - Received callback for node 69f2e18be2fa3d7693ba8c5f INFO: 172.20.0.1:36104 - "POST /node/69f2e18be2fa3d7693ba8c5f HTTP/1.0" 202 Accepted 2026-05-02 04:59:04,196 - lava_callback - INFO - Starting async job submit for node 69f2e18be2fa3d7693ba8c5f 2026-05-02 04:59:04,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:59:04,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e18be2fa3d7693ba8c5f HTTP/1.1" 200 None 2026-05-02 04:59:04,211 - lava_callback - DEBUG - Processing node 69f2e18be2fa3d7693ba8c5f: baseline-arm64-baylibre 2026-05-02 04:59:04,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e18be2fa3d7693ba8c5f 2026-05-02 04:59:04,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:59:04,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e18be2fa3d7693ba8c5f HTTP/1.1" 200 None 2026-05-02 04:59:04,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:59:04,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2024713aba9734a49455e HTTP/1.1" 200 None 2026-05-02 04:59:04,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 04:59:04,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e18be2fa3d7693ba8c5f HTTP/1.1" 200 None 2026-05-02 04:59:04,256 - lava_callback - ERROR - Error processing callback for node 69f2e18be2fa3d7693ba8c5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43916 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:00:04,307 - lava_callback - INFO - Received callback for node 69f2e1cee2fa3d7693ba8c88 INFO: 172.20.0.1:55592 - "POST /node/69f2e1cee2fa3d7693ba8c88 HTTP/1.0" 202 Accepted 2026-05-02 05:00:04,313 - lava_callback - INFO - Starting async job submit for node 69f2e1cee2fa3d7693ba8c88 2026-05-02 05:00:04,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:00:04,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e1cee2fa3d7693ba8c88 HTTP/1.1" 200 None 2026-05-02 05:00:04,328 - lava_callback - DEBUG - Processing node 69f2e1cee2fa3d7693ba8c88: baseline-arm-clabbe 2026-05-02 05:00:04,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e1cee2fa3d7693ba8c88 2026-05-02 05:00:04,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:00:04,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e1cee2fa3d7693ba8c88 HTTP/1.1" 200 None 2026-05-02 05:00:04,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:00:04,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 05:00:04,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:00:04,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e1cee2fa3d7693ba8c88 HTTP/1.1" 200 None 2026-05-02 05:00:04,375 - lava_callback - ERROR - Error processing callback for node 69f2e1cee2fa3d7693ba8c88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33368 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:01:04,281 - lava_callback - INFO - Received callback for node 69f2e1fbe2fa3d7693ba8c91 INFO: 172.20.0.1:50562 - "POST /node/69f2e1fbe2fa3d7693ba8c91 HTTP/1.0" 202 Accepted 2026-05-02 05:01:04,286 - lava_callback - INFO - Starting async job submit for node 69f2e1fbe2fa3d7693ba8c91 2026-05-02 05:01:04,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:01:04,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e1fbe2fa3d7693ba8c91 HTTP/1.1" 200 None 2026-05-02 05:01:04,314 - lava_callback - DEBUG - Processing node 69f2e1fbe2fa3d7693ba8c91: baseline-arm64-baylibre 2026-05-02 05:01:04,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e1fbe2fa3d7693ba8c91 2026-05-02 05:01:04,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:01:04,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e1fbe2fa3d7693ba8c91 HTTP/1.1" 200 None 2026-05-02 05:01:04,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:01:04,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2024713aba9734a49455e HTTP/1.1" 200 None 2026-05-02 05:01:04,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:01:04,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e1fbe2fa3d7693ba8c91 HTTP/1.1" 200 None 2026-05-02 05:01:04,360 - lava_callback - ERROR - Error processing callback for node 69f2e1fbe2fa3d7693ba8c91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39032 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:02:04,259 - lava_callback - INFO - Received callback for node 69f2e23fe2fa3d7693ba8cb5 INFO: 172.20.0.1:51316 - "POST /node/69f2e23fe2fa3d7693ba8cb5 HTTP/1.0" 202 Accepted 2026-05-02 05:02:04,265 - lava_callback - INFO - Starting async job submit for node 69f2e23fe2fa3d7693ba8cb5 2026-05-02 05:02:04,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:04,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e23fe2fa3d7693ba8cb5 HTTP/1.1" 200 None 2026-05-02 05:02:04,280 - lava_callback - DEBUG - Processing node 69f2e23fe2fa3d7693ba8cb5: baseline-arm-clabbe 2026-05-02 05:02:04,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e23fe2fa3d7693ba8cb5 2026-05-02 05:02:04,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:04,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e23fe2fa3d7693ba8cb5 HTTP/1.1" 200 None 2026-05-02 05:02:04,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:04,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 05:02:04,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:04,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e23fe2fa3d7693ba8cb5 HTTP/1.1" 200 None 2026-05-02 05:02:04,326 - lava_callback - ERROR - Error processing callback for node 69f2e23fe2fa3d7693ba8cb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59518 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:02:44,254 - lava_callback - INFO - Received callback for node 69f2e264e2fa3d7693ba8cc8 INFO: 172.20.0.1:46532 - "POST /node/69f2e264e2fa3d7693ba8cc8 HTTP/1.0" 202 Accepted 2026-05-02 05:02:44,260 - lava_callback - INFO - Starting async job submit for node 69f2e264e2fa3d7693ba8cc8 2026-05-02 05:02:44,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:44,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e264e2fa3d7693ba8cc8 HTTP/1.1" 200 None 2026-05-02 05:02:44,274 - lava_callback - DEBUG - Processing node 69f2e264e2fa3d7693ba8cc8: baseline-arm-clabbe 2026-05-02 05:02:44,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e264e2fa3d7693ba8cc8 2026-05-02 05:02:44,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:44,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e264e2fa3d7693ba8cc8 HTTP/1.1" 200 None 2026-05-02 05:02:44,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:44,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 05:02:44,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:02:44,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e264e2fa3d7693ba8cc8 HTTP/1.1" 200 None 2026-05-02 05:02:44,325 - lava_callback - ERROR - Error processing callback for node 69f2e264e2fa3d7693ba8cc8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58308 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:03:24,445 - lava_callback - INFO - Received callback for node 69f2e290e2fa3d7693ba8cd5 INFO: 172.20.0.1:43860 - "POST /node/69f2e290e2fa3d7693ba8cd5 HTTP/1.0" 202 Accepted 2026-05-02 05:03:24,451 - lava_callback - INFO - Starting async job submit for node 69f2e290e2fa3d7693ba8cd5 2026-05-02 05:03:24,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:03:24,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e290e2fa3d7693ba8cd5 HTTP/1.1" 200 None 2026-05-02 05:03:24,466 - lava_callback - DEBUG - Processing node 69f2e290e2fa3d7693ba8cd5: baseline-arm64-baylibre 2026-05-02 05:03:24,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e290e2fa3d7693ba8cd5 2026-05-02 05:03:24,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:03:24,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e290e2fa3d7693ba8cd5 HTTP/1.1" 200 None 2026-05-02 05:03:24,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:03:24,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2024713aba9734a49455e HTTP/1.1" 200 None 2026-05-02 05:03:24,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:03:24,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e290e2fa3d7693ba8cd5 HTTP/1.1" 200 None 2026-05-02 05:03:24,516 - lava_callback - ERROR - Error processing callback for node 69f2e290e2fa3d7693ba8cd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:04:24,394 - lava_callback - INFO - Received callback for node 69f2e2d3e2fa3d7693ba8cf5 INFO: 172.20.0.1:37576 - "POST /node/69f2e2d3e2fa3d7693ba8cf5 HTTP/1.0" 202 Accepted 2026-05-02 05:04:24,399 - lava_callback - INFO - Starting async job submit for node 69f2e2d3e2fa3d7693ba8cf5 2026-05-02 05:04:24,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:04:24,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e2d3e2fa3d7693ba8cf5 HTTP/1.1" 200 None 2026-05-02 05:04:24,415 - lava_callback - DEBUG - Processing node 69f2e2d3e2fa3d7693ba8cf5: baseline-arm-clabbe 2026-05-02 05:04:24,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e2d3e2fa3d7693ba8cf5 2026-05-02 05:04:24,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:04:24,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e2d3e2fa3d7693ba8cf5 HTTP/1.1" 200 None 2026-05-02 05:04:24,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:04:24,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 05:04:24,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:04:24,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e2d3e2fa3d7693ba8cf5 HTTP/1.1" 200 None 2026-05-02 05:04:24,468 - lava_callback - ERROR - Error processing callback for node 69f2e2d3e2fa3d7693ba8cf5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42786 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:05:04,446 - lava_callback - INFO - Received callback for node 69f2e2f8e2fa3d7693ba8d0c INFO: 172.20.0.1:45936 - "POST /node/69f2e2f8e2fa3d7693ba8d0c HTTP/1.0" 202 Accepted 2026-05-02 05:05:04,452 - lava_callback - INFO - Starting async job submit for node 69f2e2f8e2fa3d7693ba8d0c 2026-05-02 05:05:04,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:04,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e2f8e2fa3d7693ba8d0c HTTP/1.1" 200 None 2026-05-02 05:05:04,467 - lava_callback - DEBUG - Processing node 69f2e2f8e2fa3d7693ba8d0c: baseline-arm-clabbe 2026-05-02 05:05:04,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e2f8e2fa3d7693ba8d0c 2026-05-02 05:05:04,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:04,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e2f8e2fa3d7693ba8d0c HTTP/1.1" 200 None 2026-05-02 05:05:04,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:04,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 05:05:04,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:04,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e2f8e2fa3d7693ba8d0c HTTP/1.1" 200 None 2026-05-02 05:05:04,517 - lava_callback - ERROR - Error processing callback for node 69f2e2f8e2fa3d7693ba8d0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56712 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:05:44,444 - lava_callback - INFO - Received callback for node 69f2e31de2fa3d7693ba8d19 INFO: 172.20.0.1:60760 - "POST /node/69f2e31de2fa3d7693ba8d19 HTTP/1.0" 202 Accepted 2026-05-02 05:05:44,450 - lava_callback - INFO - Starting async job submit for node 69f2e31de2fa3d7693ba8d19 2026-05-02 05:05:44,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:44,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e31de2fa3d7693ba8d19 HTTP/1.1" 200 None 2026-05-02 05:05:44,463 - lava_callback - DEBUG - Processing node 69f2e31de2fa3d7693ba8d19: baseline-arm-clabbe 2026-05-02 05:05:44,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e31de2fa3d7693ba8d19 2026-05-02 05:05:44,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:44,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e31de2fa3d7693ba8d19 HTTP/1.1" 200 None 2026-05-02 05:05:44,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:44,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 05:05:44,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:44,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e31de2fa3d7693ba8d19 HTTP/1.1" 200 None 2026-05-02 05:05:44,513 - lava_callback - ERROR - Error processing callback for node 69f2e31de2fa3d7693ba8d19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:05:58,913 - lava_callback - INFO - Received callback for node 69f58558ed4ed40b022422fd INFO: 172.20.0.1:60064 - "POST /node/69f58558ed4ed40b022422fd HTTP/1.0" 202 Accepted 2026-05-02 05:05:58,919 - lava_callback - INFO - Starting async job submit for node 69f58558ed4ed40b022422fd 2026-05-02 05:05:58,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:58,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58558ed4ed40b022422fd HTTP/1.1" 200 None 2026-05-02 05:05:58,957 - lava_callback - DEBUG - Processing node 69f58558ed4ed40b022422fd: baseline-arm64 2026-05-02 05:05:59,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:05:59,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:05:59,401 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58558ed4ed40b022422fd/log.txt.gz 2026-05-02 05:05:59,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:05:59,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:05:59,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58558ed4ed40b022422fd/lava-logs.yaml 2026-05-02 05:05:59,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:05:59,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:05:59,848 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58558ed4ed40b022422fd/lava_callback.json.gz 2026-05-02 05:05:59,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:59,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58558ed4ed40b022422fd HTTP/1.1" 200 None 2026-05-02 05:05:59,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:59,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 05:05:59,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:05:59,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f58558ed4ed40b022422fd HTTP/1.1" 200 None 2026-05-02 05:05:59,989 - lava_callback - INFO - Completed processing callback for node 69f58558ed4ed40b022422fd INFO: 172.20.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:06:23,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:06:23,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41496 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:06:44,491 - lava_callback - INFO - Received callback for node 69f2e34fe2fa3d7693ba8d28 INFO: 172.20.0.1:59486 - "POST /node/69f2e34fe2fa3d7693ba8d28 HTTP/1.0" 202 Accepted 2026-05-02 05:06:44,497 - lava_callback - INFO - Starting async job submit for node 69f2e34fe2fa3d7693ba8d28 2026-05-02 05:06:44,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:06:44,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e34fe2fa3d7693ba8d28 HTTP/1.1" 200 None 2026-05-02 05:06:44,512 - lava_callback - DEBUG - Processing node 69f2e34fe2fa3d7693ba8d28: baseline-arm64-clabbe 2026-05-02 05:06:44,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e34fe2fa3d7693ba8d28 2026-05-02 05:06:44,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:06:44,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e34fe2fa3d7693ba8d28 HTTP/1.1" 200 None 2026-05-02 05:06:44,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:06:44,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 05:06:44,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:06:44,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e34fe2fa3d7693ba8d28 HTTP/1.1" 200 None 2026-05-02 05:06:44,562 - lava_callback - ERROR - Error processing callback for node 69f2e34fe2fa3d7693ba8d28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43522 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:07:13,891 - lava_callback - INFO - Received callback for node 69f55ee7ed4ed40b02241221 INFO: 172.20.0.1:53444 - "POST /node/69f55ee7ed4ed40b02241221 HTTP/1.0" 202 Accepted 2026-05-02 05:07:13,898 - lava_callback - INFO - Starting async job submit for node 69f55ee7ed4ed40b02241221 2026-05-02 05:07:13,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:13,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55ee7ed4ed40b02241221 HTTP/1.1" 200 None 2026-05-02 05:07:13,940 - lava_callback - DEBUG - Processing node 69f55ee7ed4ed40b02241221: baseline-nfs-arm64 2026-05-02 05:07:14,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:07:14,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:07:14,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f55ee7ed4ed40b02241221/log.txt.gz 2026-05-02 05:07:14,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:07:15,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:07:15,050 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f55ee7ed4ed40b02241221/lava-logs.yaml 2026-05-02 05:07:15,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:53450 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 05:07:15,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:07:15,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f55ee7ed4ed40b02241221/lava_callback.json.gz 2026-05-02 05:07:15,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:15,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f55ee7ed4ed40b02241221 HTTP/1.1" 200 None 2026-05-02 05:07:15,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:15,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 05:07:15,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:15,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f55ee7ed4ed40b02241221 HTTP/1.1" 200 None 2026-05-02 05:07:15,352 - lava_callback - INFO - Completed processing callback for node 69f55ee7ed4ed40b02241221 INFO: 172.20.0.1:53466 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:07:23,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:23,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54720 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:07:44,508 - lava_callback - INFO - Received callback for node 69f2e38de2fa3d7693ba8d49 INFO: 172.20.0.1:53794 - "POST /node/69f2e38de2fa3d7693ba8d49 HTTP/1.0" 202 Accepted 2026-05-02 05:07:44,513 - lava_callback - INFO - Starting async job submit for node 69f2e38de2fa3d7693ba8d49 2026-05-02 05:07:44,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:44,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e38de2fa3d7693ba8d49 HTTP/1.1" 200 None 2026-05-02 05:07:44,529 - lava_callback - DEBUG - Processing node 69f2e38de2fa3d7693ba8d49: baseline-arm-clabbe 2026-05-02 05:07:44,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e38de2fa3d7693ba8d49 2026-05-02 05:07:44,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:44,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e38de2fa3d7693ba8d49 HTTP/1.1" 200 None 2026-05-02 05:07:44,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:44,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 05:07:44,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:07:44,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e38de2fa3d7693ba8d49 HTTP/1.1" 200 None 2026-05-02 05:07:44,590 - lava_callback - ERROR - Error processing callback for node 69f2e38de2fa3d7693ba8d49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:08:44,616 - lava_callback - INFO - Received callback for node 69f2e3c6e2fa3d7693ba8d61 INFO: 172.20.0.1:33428 - "POST /node/69f2e3c6e2fa3d7693ba8d61 HTTP/1.0" 202 Accepted 2026-05-02 05:08:44,622 - lava_callback - INFO - Starting async job submit for node 69f2e3c6e2fa3d7693ba8d61 2026-05-02 05:08:44,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:08:44,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e3c6e2fa3d7693ba8d61 HTTP/1.1" 200 None 2026-05-02 05:08:44,645 - lava_callback - DEBUG - Processing node 69f2e3c6e2fa3d7693ba8d61: baseline-arm64-clabbe 2026-05-02 05:08:44,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e3c6e2fa3d7693ba8d61 2026-05-02 05:08:44,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:08:44,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e3c6e2fa3d7693ba8d61 HTTP/1.1" 200 None 2026-05-02 05:08:44,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:08:44,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5dc08305534e972331e6 HTTP/1.1" 200 None 2026-05-02 05:08:44,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:08:44,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e3c6e2fa3d7693ba8d61 HTTP/1.1" 200 None 2026-05-02 05:08:44,692 - lava_callback - ERROR - Error processing callback for node 69f2e3c6e2fa3d7693ba8d61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37286 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:09:44,707 - lava_callback - INFO - Received callback for node 69f2e40ce2fa3d7693ba8d8a INFO: 172.20.0.1:43474 - "POST /node/69f2e40ce2fa3d7693ba8d8a HTTP/1.0" 202 Accepted 2026-05-02 05:09:44,712 - lava_callback - INFO - Starting async job submit for node 69f2e40ce2fa3d7693ba8d8a 2026-05-02 05:09:44,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:09:44,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e40ce2fa3d7693ba8d8a HTTP/1.1" 200 None 2026-05-02 05:09:44,728 - lava_callback - DEBUG - Processing node 69f2e40ce2fa3d7693ba8d8a: baseline-arm64-baylibre 2026-05-02 05:09:44,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e40ce2fa3d7693ba8d8a 2026-05-02 05:09:44,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:09:44,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e40ce2fa3d7693ba8d8a HTTP/1.1" 200 None 2026-05-02 05:09:44,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:09:44,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:09:44,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:09:44,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e40ce2fa3d7693ba8d8a HTTP/1.1" 200 None 2026-05-02 05:09:44,776 - lava_callback - ERROR - Error processing callback for node 69f2e40ce2fa3d7693ba8d8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49452 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:11:04,775 - lava_callback - INFO - Received callback for node 69f2e45de2fa3d7693ba8dac INFO: 172.20.0.1:51426 - "POST /node/69f2e45de2fa3d7693ba8dac HTTP/1.0" 202 Accepted 2026-05-02 05:11:04,781 - lava_callback - INFO - Starting async job submit for node 69f2e45de2fa3d7693ba8dac 2026-05-02 05:11:04,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:11:04,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e45de2fa3d7693ba8dac HTTP/1.1" 200 None 2026-05-02 05:11:04,796 - lava_callback - DEBUG - Processing node 69f2e45de2fa3d7693ba8dac: baseline-arm64-baylibre 2026-05-02 05:11:04,796 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e45de2fa3d7693ba8dac 2026-05-02 05:11:04,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:11:04,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e45de2fa3d7693ba8dac HTTP/1.1" 200 None 2026-05-02 05:11:04,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:11:04,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:11:04,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:11:04,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e45de2fa3d7693ba8dac HTTP/1.1" 200 None 2026-05-02 05:11:04,841 - lava_callback - ERROR - Error processing callback for node 69f2e45de2fa3d7693ba8dac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33084 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:12:24,931 - lava_callback - INFO - Received callback for node 69f2e4ace2fa3d7693ba8dd9 INFO: 172.20.0.1:53424 - "POST /node/69f2e4ace2fa3d7693ba8dd9 HTTP/1.0" 202 Accepted 2026-05-02 05:12:24,939 - lava_callback - INFO - Starting async job submit for node 69f2e4ace2fa3d7693ba8dd9 2026-05-02 05:12:24,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:12:24,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e4ace2fa3d7693ba8dd9 HTTP/1.1" 200 None 2026-05-02 05:12:24,958 - lava_callback - DEBUG - Processing node 69f2e4ace2fa3d7693ba8dd9: baseline-arm64-baylibre 2026-05-02 05:12:24,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e4ace2fa3d7693ba8dd9 2026-05-02 05:12:24,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:12:24,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e4ace2fa3d7693ba8dd9 HTTP/1.1" 200 None 2026-05-02 05:12:24,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:12:24,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:12:24,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:12:25,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e4ace2fa3d7693ba8dd9 HTTP/1.1" 200 None 2026-05-02 05:12:25,015 - lava_callback - ERROR - Error processing callback for node 69f2e4ace2fa3d7693ba8dd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34746 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:13:45,014 - lava_callback - INFO - Received callback for node 69f2e4fae2fa3d7693ba8e05 INFO: 172.20.0.1:54136 - "POST /node/69f2e4fae2fa3d7693ba8e05 HTTP/1.0" 202 Accepted 2026-05-02 05:13:45,022 - lava_callback - INFO - Starting async job submit for node 69f2e4fae2fa3d7693ba8e05 2026-05-02 05:13:45,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:45,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e4fae2fa3d7693ba8e05 HTTP/1.1" 200 None 2026-05-02 05:13:45,038 - lava_callback - DEBUG - Processing node 69f2e4fae2fa3d7693ba8e05: baseline-arm64-baylibre 2026-05-02 05:13:45,038 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e4fae2fa3d7693ba8e05 2026-05-02 05:13:45,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:45,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e4fae2fa3d7693ba8e05 HTTP/1.1" 200 None 2026-05-02 05:13:45,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:45,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:13:45,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:45,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e4fae2fa3d7693ba8e05 HTTP/1.1" 200 None 2026-05-02 05:13:45,085 - lava_callback - ERROR - Error processing callback for node 69f2e4fae2fa3d7693ba8e05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54142 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:13:55,241 - lava_callback - INFO - Received callback for node 69f58409ed4ed40b02242264 INFO: 172.20.0.1:51592 - "POST /node/69f58409ed4ed40b02242264 HTTP/1.0" 202 Accepted 2026-05-02 05:13:55,248 - lava_callback - INFO - Starting async job submit for node 69f58409ed4ed40b02242264 2026-05-02 05:13:55,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:55,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58409ed4ed40b02242264 HTTP/1.1" 200 None 2026-05-02 05:13:55,320 - lava_callback - DEBUG - Processing node 69f58409ed4ed40b02242264: ltp-smoke 2026-05-02 05:13:55,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:13:55,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:13:55,945 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f58409ed4ed40b02242264/log.txt.gz 2026-05-02 05:13:56,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:13:56,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:13:56,424 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f58409ed4ed40b02242264/lava-logs.yaml 2026-05-02 05:13:56,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51608 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:13:56,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:13:56,555 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f58409ed4ed40b02242264/lava_callback.json.gz 2026-05-02 05:13:56,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:56,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58409ed4ed40b02242264 HTTP/1.1" 200 None 2026-05-02 05:13:56,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:56,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f54464ed4ed40b0223db05 HTTP/1.1" 200 None 2026-05-02 05:13:56,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:13:56,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f58409ed4ed40b02242264 HTTP/1.1" 200 None 2026-05-02 05:13:56,837 - lava_callback - INFO - Completed processing callback for node 69f58409ed4ed40b02242264 INFO: 172.20.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:14:23,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:14:23,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39202 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:15:05,092 - lava_callback - INFO - Received callback for node 69f2e548e2fa3d7693ba8e27 INFO: 172.20.0.1:58654 - "POST /node/69f2e548e2fa3d7693ba8e27 HTTP/1.0" 202 Accepted 2026-05-02 05:15:05,098 - lava_callback - INFO - Starting async job submit for node 69f2e548e2fa3d7693ba8e27 2026-05-02 05:15:05,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:15:05,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e548e2fa3d7693ba8e27 HTTP/1.1" 200 None 2026-05-02 05:15:05,113 - lava_callback - DEBUG - Processing node 69f2e548e2fa3d7693ba8e27: baseline-arm64-baylibre 2026-05-02 05:15:05,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e548e2fa3d7693ba8e27 2026-05-02 05:15:05,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:15:05,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e548e2fa3d7693ba8e27 HTTP/1.1" 200 None 2026-05-02 05:15:05,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:15:05,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:15:05,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:15:05,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e548e2fa3d7693ba8e27 HTTP/1.1" 200 None 2026-05-02 05:15:05,158 - lava_callback - ERROR - Error processing callback for node 69f2e548e2fa3d7693ba8e27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34310 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:17:25,200 - lava_callback - INFO - Received callback for node 69f2e5e2e2fa3d7693ba8e72 INFO: 172.20.0.1:34898 - "POST /node/69f2e5e2e2fa3d7693ba8e72 HTTP/1.0" 202 Accepted 2026-05-02 05:17:25,205 - lava_callback - INFO - Starting async job submit for node 69f2e5e2e2fa3d7693ba8e72 2026-05-02 05:17:25,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:25,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e5e2e2fa3d7693ba8e72 HTTP/1.1" 200 None 2026-05-02 05:17:25,230 - lava_callback - DEBUG - Processing node 69f2e5e2e2fa3d7693ba8e72: baseline-arm64-baylibre 2026-05-02 05:17:25,231 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e5e2e2fa3d7693ba8e72 2026-05-02 05:17:25,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:25,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e5e2e2fa3d7693ba8e72 HTTP/1.1" 200 None 2026-05-02 05:17:25,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:25,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:17:25,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:25,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e5e2e2fa3d7693ba8e72 HTTP/1.1" 200 None 2026-05-02 05:17:25,274 - lava_callback - ERROR - Error processing callback for node 69f2e5e2e2fa3d7693ba8e72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:17:52,616 - lava_callback - INFO - Received callback for node 69f584e9ed4ed40b022422cf INFO: 172.20.0.1:38412 - "POST /node/69f584e9ed4ed40b022422cf HTTP/1.0" 202 Accepted 2026-05-02 05:17:52,623 - lava_callback - INFO - Starting async job submit for node 69f584e9ed4ed40b022422cf 2026-05-02 05:17:52,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:52,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f584e9ed4ed40b022422cf HTTP/1.1" 200 None 2026-05-02 05:17:52,696 - lava_callback - DEBUG - Processing node 69f584e9ed4ed40b022422cf: baseline-arm 2026-05-02 05:17:53,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:17:53,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:17:53,375 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f584e9ed4ed40b022422cf/log.txt.gz 2026-05-02 05:17:53,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:17:53,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:17:53,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f584e9ed4ed40b022422cf/lava-logs.yaml 2026-05-02 05:17:53,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:17:54,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:17:54,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f584e9ed4ed40b022422cf/lava_callback.json.gz DEBUG: Setup failed for 69f584e9ed4ed40b022422cf. Transitting job node result: incomplete -> fail 2026-05-02 05:17:54,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:54,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f584e9ed4ed40b022422cf HTTP/1.1" 200 None 2026-05-02 05:17:54,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:54,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5445fed4ed40b0223daff HTTP/1.1" 200 None 2026-05-02 05:17:54,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:17:54,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f584e9ed4ed40b022422cf HTTP/1.1" 200 None 2026-05-02 05:17:54,514 - lava_callback - INFO - Completed processing callback for node 69f584e9ed4ed40b022422cf INFO: 172.20.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:18:23,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:18:23,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:18:45,403 - lava_callback - INFO - Received callback for node 69f2e630e2fa3d7693ba8e9f INFO: 172.20.0.1:55352 - "POST /node/69f2e630e2fa3d7693ba8e9f HTTP/1.0" 202 Accepted 2026-05-02 05:18:45,409 - lava_callback - INFO - Starting async job submit for node 69f2e630e2fa3d7693ba8e9f 2026-05-02 05:18:45,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:18:45,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e630e2fa3d7693ba8e9f HTTP/1.1" 200 None 2026-05-02 05:18:45,426 - lava_callback - DEBUG - Processing node 69f2e630e2fa3d7693ba8e9f: baseline-arm64-baylibre 2026-05-02 05:18:45,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e630e2fa3d7693ba8e9f 2026-05-02 05:18:45,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:18:45,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e630e2fa3d7693ba8e9f HTTP/1.1" 200 None 2026-05-02 05:18:45,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:18:45,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:18:45,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:18:45,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e630e2fa3d7693ba8e9f HTTP/1.1" 200 None 2026-05-02 05:18:45,472 - lava_callback - ERROR - Error processing callback for node 69f2e630e2fa3d7693ba8e9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53576 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:20:05,331 - lava_callback - INFO - Received callback for node 69f2e678e2fa3d7693ba8ec0 INFO: 172.20.0.1:59190 - "POST /node/69f2e678e2fa3d7693ba8ec0 HTTP/1.0" 202 Accepted 2026-05-02 05:20:05,337 - lava_callback - INFO - Starting async job submit for node 69f2e678e2fa3d7693ba8ec0 2026-05-02 05:20:05,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:20:05,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e678e2fa3d7693ba8ec0 HTTP/1.1" 200 None 2026-05-02 05:20:05,360 - lava_callback - DEBUG - Processing node 69f2e678e2fa3d7693ba8ec0: baseline-arm64-baylibre 2026-05-02 05:20:05,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e678e2fa3d7693ba8ec0 2026-05-02 05:20:05,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:20:05,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e678e2fa3d7693ba8ec0 HTTP/1.1" 200 None 2026-05-02 05:20:05,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:20:05,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:20:05,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:20:05,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e678e2fa3d7693ba8ec0 HTTP/1.1" 200 None 2026-05-02 05:20:05,404 - lava_callback - ERROR - Error processing callback for node 69f2e678e2fa3d7693ba8ec0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:21:45,481 - lava_callback - INFO - Received callback for node 69f2e6e3e2fa3d7693ba8ef7 INFO: 172.20.0.1:46686 - "POST /node/69f2e6e3e2fa3d7693ba8ef7 HTTP/1.0" 202 Accepted 2026-05-02 05:21:45,487 - lava_callback - INFO - Starting async job submit for node 69f2e6e3e2fa3d7693ba8ef7 2026-05-02 05:21:45,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:21:45,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e6e3e2fa3d7693ba8ef7 HTTP/1.1" 200 None 2026-05-02 05:21:45,502 - lava_callback - DEBUG - Processing node 69f2e6e3e2fa3d7693ba8ef7: baseline-arm-baylibre 2026-05-02 05:21:45,502 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e6e3e2fa3d7693ba8ef7 2026-05-02 05:21:45,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:21:45,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e6e3e2fa3d7693ba8ef7 HTTP/1.1" 200 None 2026-05-02 05:21:45,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:21:45,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:21:45,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:21:45,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e6e3e2fa3d7693ba8ef7 HTTP/1.1" 200 None 2026-05-02 05:21:45,570 - lava_callback - ERROR - Error processing callback for node 69f2e6e3e2fa3d7693ba8ef7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33872 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:22:25,426 - lava_callback - INFO - Received callback for node 69f2e709e2fa3d7693ba8f05 INFO: 172.20.0.1:49658 - "POST /node/69f2e709e2fa3d7693ba8f05 HTTP/1.0" 202 Accepted 2026-05-02 05:22:25,431 - lava_callback - INFO - Starting async job submit for node 69f2e709e2fa3d7693ba8f05 2026-05-02 05:22:25,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:22:25,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e709e2fa3d7693ba8f05 HTTP/1.1" 200 None 2026-05-02 05:22:25,447 - lava_callback - DEBUG - Processing node 69f2e709e2fa3d7693ba8f05: baseline-arm-baylibre 2026-05-02 05:22:25,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e709e2fa3d7693ba8f05 2026-05-02 05:22:25,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:22:25,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e709e2fa3d7693ba8f05 HTTP/1.1" 200 None 2026-05-02 05:22:25,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:22:25,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:22:25,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:22:25,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e709e2fa3d7693ba8f05 HTTP/1.1" 200 None 2026-05-02 05:22:25,495 - lava_callback - ERROR - Error processing callback for node 69f2e709e2fa3d7693ba8f05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:23:25,608 - lava_callback - INFO - Received callback for node 69f2e73de2fa3d7693ba8f16 INFO: 172.20.0.1:54182 - "POST /node/69f2e73de2fa3d7693ba8f16 HTTP/1.0" 202 Accepted 2026-05-02 05:23:25,614 - lava_callback - INFO - Starting async job submit for node 69f2e73de2fa3d7693ba8f16 2026-05-02 05:23:25,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:23:25,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e73de2fa3d7693ba8f16 HTTP/1.1" 200 None 2026-05-02 05:23:25,635 - lava_callback - DEBUG - Processing node 69f2e73de2fa3d7693ba8f16: baseline-arm64-baylibre 2026-05-02 05:23:25,635 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e73de2fa3d7693ba8f16 2026-05-02 05:23:25,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:23:25,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e73de2fa3d7693ba8f16 HTTP/1.1" 200 None 2026-05-02 05:23:25,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:23:25,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:23:25,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:23:25,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e73de2fa3d7693ba8f16 HTTP/1.1" 200 None 2026-05-02 05:23:25,680 - lava_callback - ERROR - Error processing callback for node 69f2e73de2fa3d7693ba8f16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53294 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:24:25,540 - lava_callback - INFO - Received callback for node 69f2e780e2fa3d7693ba8f31 INFO: 172.20.0.1:54986 - "POST /node/69f2e780e2fa3d7693ba8f31 HTTP/1.0" 202 Accepted 2026-05-02 05:24:25,546 - lava_callback - INFO - Starting async job submit for node 69f2e780e2fa3d7693ba8f31 2026-05-02 05:24:25,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:24:25,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e780e2fa3d7693ba8f31 HTTP/1.1" 200 None 2026-05-02 05:24:25,561 - lava_callback - DEBUG - Processing node 69f2e780e2fa3d7693ba8f31: baseline-arm-baylibre 2026-05-02 05:24:25,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e780e2fa3d7693ba8f31 2026-05-02 05:24:25,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:24:25,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e780e2fa3d7693ba8f31 HTTP/1.1" 200 None 2026-05-02 05:24:25,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:54998 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:24:25,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:24:25,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:24:25,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e780e2fa3d7693ba8f31 HTTP/1.1" 200 None 2026-05-02 05:24:25,612 - lava_callback - ERROR - Error processing callback for node 69f2e780e2fa3d7693ba8f31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:25:05,623 - lava_callback - INFO - Received callback for node 69f2e7a7e2fa3d7693ba8f44 INFO: 172.20.0.1:53868 - "POST /node/69f2e7a7e2fa3d7693ba8f44 HTTP/1.0" 202 Accepted 2026-05-02 05:25:05,629 - lava_callback - INFO - Starting async job submit for node 69f2e7a7e2fa3d7693ba8f44 2026-05-02 05:25:05,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:05,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e7a7e2fa3d7693ba8f44 HTTP/1.1" 200 None 2026-05-02 05:25:05,648 - lava_callback - DEBUG - Processing node 69f2e7a7e2fa3d7693ba8f44: baseline-arm-baylibre 2026-05-02 05:25:05,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e7a7e2fa3d7693ba8f44 2026-05-02 05:25:05,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:05,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e7a7e2fa3d7693ba8f44 HTTP/1.1" 200 None 2026-05-02 05:25:05,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:05,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:25:05,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:05,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e7a7e2fa3d7693ba8f44 HTTP/1.1" 200 None 2026-05-02 05:25:05,694 - lava_callback - ERROR - Error processing callback for node 69f2e7a7e2fa3d7693ba8f44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34720 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:25:45,605 - lava_callback - INFO - Received callback for node 69f2e7d7e2fa3d7693ba8f50 INFO: 172.20.0.1:58070 - "POST /node/69f2e7d7e2fa3d7693ba8f50 HTTP/1.0" 202 Accepted 2026-05-02 05:25:45,614 - lava_callback - INFO - Starting async job submit for node 69f2e7d7e2fa3d7693ba8f50 2026-05-02 05:25:45,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:45,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e7d7e2fa3d7693ba8f50 HTTP/1.1" 200 None 2026-05-02 05:25:45,633 - lava_callback - DEBUG - Processing node 69f2e7d7e2fa3d7693ba8f50: baseline-arm64-baylibre 2026-05-02 05:25:45,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e7d7e2fa3d7693ba8f50 2026-05-02 05:25:45,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:45,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e7d7e2fa3d7693ba8f50 HTTP/1.1" 200 None 2026-05-02 05:25:45,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:45,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:25:45,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:25:45,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e7d7e2fa3d7693ba8f50 HTTP/1.1" 200 None 2026-05-02 05:25:45,687 - lava_callback - ERROR - Error processing callback for node 69f2e7d7e2fa3d7693ba8f50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45310 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:28:05,777 - lava_callback - INFO - Received callback for node 69f2e864e2fa3d7693ba8f9c INFO: 172.20.0.1:56568 - "POST /node/69f2e864e2fa3d7693ba8f9c HTTP/1.0" 202 Accepted 2026-05-02 05:28:05,783 - lava_callback - INFO - Starting async job submit for node 69f2e864e2fa3d7693ba8f9c 2026-05-02 05:28:05,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:28:05,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e864e2fa3d7693ba8f9c HTTP/1.1" 200 None 2026-05-02 05:28:05,806 - lava_callback - DEBUG - Processing node 69f2e864e2fa3d7693ba8f9c: baseline-arm-baylibre 2026-05-02 05:28:05,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e864e2fa3d7693ba8f9c 2026-05-02 05:28:05,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:28:05,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e864e2fa3d7693ba8f9c HTTP/1.1" 200 None 2026-05-02 05:28:05,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:28:05,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:28:05,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:28:05,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e864e2fa3d7693ba8f9c HTTP/1.1" 200 None 2026-05-02 05:28:05,854 - lava_callback - ERROR - Error processing callback for node 69f2e864e2fa3d7693ba8f9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33266 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:29:05,773 - lava_callback - INFO - Received callback for node 69f2e892e2fa3d7693ba8faa INFO: 172.20.0.1:43604 - "POST /node/69f2e892e2fa3d7693ba8faa HTTP/1.0" 202 Accepted 2026-05-02 05:29:05,782 - lava_callback - INFO - Starting async job submit for node 69f2e892e2fa3d7693ba8faa 2026-05-02 05:29:05,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:29:05,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e892e2fa3d7693ba8faa HTTP/1.1" 200 None 2026-05-02 05:29:05,815 - lava_callback - DEBUG - Processing node 69f2e892e2fa3d7693ba8faa: baseline-arm64-baylibre 2026-05-02 05:29:05,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e892e2fa3d7693ba8faa 2026-05-02 05:29:05,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:29:05,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e892e2fa3d7693ba8faa HTTP/1.1" 200 None 2026-05-02 05:29:05,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:29:05,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:29:05,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:29:05,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e892e2fa3d7693ba8faa HTTP/1.1" 200 None 2026-05-02 05:29:05,862 - lava_callback - ERROR - Error processing callback for node 69f2e892e2fa3d7693ba8faa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51794 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:30:05,906 - lava_callback - INFO - Received callback for node 69f2e8d3e2fa3d7693ba8fcb INFO: 172.20.0.1:40982 - "POST /node/69f2e8d3e2fa3d7693ba8fcb HTTP/1.0" 202 Accepted 2026-05-02 05:30:05,913 - lava_callback - INFO - Starting async job submit for node 69f2e8d3e2fa3d7693ba8fcb 2026-05-02 05:30:05,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:05,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e8d3e2fa3d7693ba8fcb HTTP/1.1" 200 None 2026-05-02 05:30:05,932 - lava_callback - DEBUG - Processing node 69f2e8d3e2fa3d7693ba8fcb: baseline-arm-baylibre 2026-05-02 05:30:05,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e8d3e2fa3d7693ba8fcb 2026-05-02 05:30:05,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:05,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e8d3e2fa3d7693ba8fcb HTTP/1.1" 200 None 2026-05-02 05:30:05,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:05,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:30:05,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:05,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e8d3e2fa3d7693ba8fcb HTTP/1.1" 200 None 2026-05-02 05:30:05,984 - lava_callback - ERROR - Error processing callback for node 69f2e8d3e2fa3d7693ba8fcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57460 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:30:45,863 - lava_callback - INFO - Received callback for node 69f2e900e2fa3d7693ba8fdf INFO: 172.20.0.1:56428 - "POST /node/69f2e900e2fa3d7693ba8fdf HTTP/1.0" 202 Accepted 2026-05-02 05:30:45,868 - lava_callback - INFO - Starting async job submit for node 69f2e900e2fa3d7693ba8fdf 2026-05-02 05:30:45,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:45,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e900e2fa3d7693ba8fdf HTTP/1.1" 200 None 2026-05-02 05:30:45,884 - lava_callback - DEBUG - Processing node 69f2e900e2fa3d7693ba8fdf: baseline-arm64-baylibre 2026-05-02 05:30:45,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e900e2fa3d7693ba8fdf 2026-05-02 05:30:45,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:45,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e900e2fa3d7693ba8fdf HTTP/1.1" 200 None 2026-05-02 05:30:45,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:45,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:30:45,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:30:45,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e900e2fa3d7693ba8fdf HTTP/1.1" 200 None 2026-05-02 05:30:45,928 - lava_callback - ERROR - Error processing callback for node 69f2e900e2fa3d7693ba8fdf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:32:05,741 - lava_callback - INFO - Received callback for node 69f2e94ee2fa3d7693ba8ffe INFO: 172.20.0.1:41612 - "POST /node/69f2e94ee2fa3d7693ba8ffe HTTP/1.0" 202 Accepted 2026-05-02 05:32:05,747 - lava_callback - INFO - Starting async job submit for node 69f2e94ee2fa3d7693ba8ffe 2026-05-02 05:32:05,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:05,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e94ee2fa3d7693ba8ffe HTTP/1.1" 200 None 2026-05-02 05:32:05,767 - lava_callback - DEBUG - Processing node 69f2e94ee2fa3d7693ba8ffe: baseline-arm64-clabbe 2026-05-02 05:32:05,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e94ee2fa3d7693ba8ffe 2026-05-02 05:32:05,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:05,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e94ee2fa3d7693ba8ffe HTTP/1.1" 200 None 2026-05-02 05:32:05,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:05,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:32:05,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:05,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e94ee2fa3d7693ba8ffe HTTP/1.1" 200 None 2026-05-02 05:32:05,813 - lava_callback - ERROR - Error processing callback for node 69f2e94ee2fa3d7693ba8ffe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:32:42,923 - lava_callback - INFO - Received callback for node 69f58510ed4ed40b022422dd INFO: 172.20.0.1:59888 - "POST /node/69f58510ed4ed40b022422dd HTTP/1.0" 202 Accepted 2026-05-02 05:32:42,930 - lava_callback - INFO - Starting async job submit for node 69f58510ed4ed40b022422dd 2026-05-02 05:32:42,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:42,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58510ed4ed40b022422dd HTTP/1.1" 200 None 2026-05-02 05:32:42,987 - lava_callback - DEBUG - Processing node 69f58510ed4ed40b022422dd: baseline-arm64 2026-05-02 05:32:43,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:32:43,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:32:43,595 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58510ed4ed40b022422dd/log.txt.gz 2026-05-02 05:32:43,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:32:44,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:32:44,068 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58510ed4ed40b022422dd/lava-logs.yaml 2026-05-02 05:32:44,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:32:44,234 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:32:44,235 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f58510ed4ed40b022422dd/lava_callback.json.gz 2026-05-02 05:32:44,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:44,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58510ed4ed40b022422dd HTTP/1.1" 200 None 2026-05-02 05:32:44,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:44,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 05:32:44,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:44,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f58510ed4ed40b022422dd HTTP/1.1" 200 None 2026-05-02 05:32:44,358 - lava_callback - INFO - Completed processing callback for node 69f58510ed4ed40b022422dd INFO: 172.20.0.1:59894 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:32:53,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:32:53,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:33:25,669 - lava_callback - INFO - Received callback for node 69f2e999e2fa3d7693ba902d INFO: 172.20.0.1:50580 - "POST /node/69f2e999e2fa3d7693ba902d HTTP/1.0" 202 Accepted 2026-05-02 05:33:25,675 - lava_callback - INFO - Starting async job submit for node 69f2e999e2fa3d7693ba902d 2026-05-02 05:33:25,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:25,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e999e2fa3d7693ba902d HTTP/1.1" 200 None 2026-05-02 05:33:25,692 - lava_callback - DEBUG - Processing node 69f2e999e2fa3d7693ba902d: baseline-arm64-clabbe 2026-05-02 05:33:25,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e999e2fa3d7693ba902d 2026-05-02 05:33:25,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:25,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e999e2fa3d7693ba902d HTTP/1.1" 200 None 2026-05-02 05:33:25,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:25,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:33:25,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:50586 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:33:25,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e999e2fa3d7693ba902d HTTP/1.1" 200 None 2026-05-02 05:33:25,748 - lava_callback - ERROR - Error processing callback for node 69f2e999e2fa3d7693ba902d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50590 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:33:33,371 - lava_callback - INFO - Received callback for node 69f58808ed4ed40b0224244b INFO: 172.20.0.1:53452 - "POST /node/69f58808ed4ed40b0224244b HTTP/1.0" 202 Accepted 2026-05-02 05:33:33,379 - lava_callback - INFO - Starting async job submit for node 69f58808ed4ed40b0224244b 2026-05-02 05:33:33,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:33,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58808ed4ed40b0224244b HTTP/1.1" 200 None 2026-05-02 05:33:33,443 - lava_callback - DEBUG - Processing node 69f58808ed4ed40b0224244b: baseline-nfs-arm64 2026-05-02 05:33:34,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:33:34,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:33:34,678 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f58808ed4ed40b0224244b/log.txt.gz 2026-05-02 05:33:35,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:33:35,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:33:35,544 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f58808ed4ed40b0224244b/lava-logs.yaml 2026-05-02 05:33:35,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:33:35,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:33:35,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f58808ed4ed40b0224244b/lava_callback.json.gz DEBUG: Setup failed for 69f58808ed4ed40b0224244b. Transitting job node result: incomplete -> fail 2026-05-02 05:33:35,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:35,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58808ed4ed40b0224244b HTTP/1.1" 200 None 2026-05-02 05:33:35,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:35,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 05:33:35,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:35,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f58808ed4ed40b0224244b HTTP/1.1" 200 None 2026-05-02 05:33:35,979 - lava_callback - INFO - Completed processing callback for node 69f58808ed4ed40b0224244b INFO: 172.20.0.1:53462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43614 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:33:53,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:33:53,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:34:45,746 - lava_callback - INFO - Received callback for node 69f2e9e5e2fa3d7693ba905e INFO: 172.20.0.1:38668 - "POST /node/69f2e9e5e2fa3d7693ba905e HTTP/1.0" 202 Accepted 2026-05-02 05:34:45,752 - lava_callback - INFO - Starting async job submit for node 69f2e9e5e2fa3d7693ba905e 2026-05-02 05:34:45,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:34:45,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e9e5e2fa3d7693ba905e HTTP/1.1" 200 None 2026-05-02 05:34:45,767 - lava_callback - DEBUG - Processing node 69f2e9e5e2fa3d7693ba905e: baseline-arm64-clabbe 2026-05-02 05:34:45,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2e9e5e2fa3d7693ba905e 2026-05-02 05:34:45,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:34:45,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2e9e5e2fa3d7693ba905e HTTP/1.1" 200 None 2026-05-02 05:34:45,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:34:45,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:34:45,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:34:45,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2e9e5e2fa3d7693ba905e HTTP/1.1" 200 None 2026-05-02 05:34:45,830 - lava_callback - ERROR - Error processing callback for node 69f2e9e5e2fa3d7693ba905e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53088 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:35:46,173 - lava_callback - INFO - Received callback for node 69f2ea29e2fa3d7693ba9078 INFO: 172.20.0.1:55138 - "POST /node/69f2ea29e2fa3d7693ba9078 HTTP/1.0" 202 Accepted 2026-05-02 05:35:46,179 - lava_callback - INFO - Starting async job submit for node 69f2ea29e2fa3d7693ba9078 2026-05-02 05:35:46,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:35:46,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ea29e2fa3d7693ba9078 HTTP/1.1" 200 None 2026-05-02 05:35:46,195 - lava_callback - DEBUG - Processing node 69f2ea29e2fa3d7693ba9078: baseline-arm64-baylibre 2026-05-02 05:35:46,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ea29e2fa3d7693ba9078 2026-05-02 05:35:46,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:35:46,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ea29e2fa3d7693ba9078 HTTP/1.1" 200 None 2026-05-02 05:35:46,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:35:46,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 05:35:46,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:35:46,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ea29e2fa3d7693ba9078 HTTP/1.1" 200 None 2026-05-02 05:35:46,247 - lava_callback - ERROR - Error processing callback for node 69f2ea29e2fa3d7693ba9078: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51196 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:37:05,865 - lava_callback - INFO - Received callback for node 69f2ea7ce2fa3d7693ba90ae INFO: 172.20.0.1:39028 - "POST /node/69f2ea7ce2fa3d7693ba90ae HTTP/1.0" 202 Accepted 2026-05-02 05:37:05,870 - lava_callback - INFO - Starting async job submit for node 69f2ea7ce2fa3d7693ba90ae 2026-05-02 05:37:05,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:37:05,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ea7ce2fa3d7693ba90ae HTTP/1.1" 200 None 2026-05-02 05:37:05,885 - lava_callback - DEBUG - Processing node 69f2ea7ce2fa3d7693ba90ae: baseline-arm64-clabbe 2026-05-02 05:37:05,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ea7ce2fa3d7693ba90ae 2026-05-02 05:37:05,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:37:05,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ea7ce2fa3d7693ba90ae HTTP/1.1" 200 None 2026-05-02 05:37:05,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:37:05,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:37:05,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:37:05,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ea7ce2fa3d7693ba90ae HTTP/1.1" 200 None 2026-05-02 05:37:05,942 - lava_callback - ERROR - Error processing callback for node 69f2ea7ce2fa3d7693ba90ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33402 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:38:26,333 - lava_callback - INFO - Received callback for node 69f2eabfe2fa3d7693ba90ce INFO: 172.20.0.1:33412 - "POST /node/69f2eabfe2fa3d7693ba90ce HTTP/1.0" 202 Accepted 2026-05-02 05:38:26,339 - lava_callback - INFO - Starting async job submit for node 69f2eabfe2fa3d7693ba90ce 2026-05-02 05:38:26,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:38:26,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eabfe2fa3d7693ba90ce HTTP/1.1" 200 None 2026-05-02 05:38:26,354 - lava_callback - DEBUG - Processing node 69f2eabfe2fa3d7693ba90ce: baseline-arm64-baylibre 2026-05-02 05:38:26,354 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2eabfe2fa3d7693ba90ce 2026-05-02 05:38:26,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:38:26,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eabfe2fa3d7693ba90ce HTTP/1.1" 200 None 2026-05-02 05:38:26,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:38:26,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 05:38:26,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:38:26,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2eabfe2fa3d7693ba90ce HTTP/1.1" 200 None 2026-05-02 05:38:26,416 - lava_callback - ERROR - Error processing callback for node 69f2eabfe2fa3d7693ba90ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42648 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:39:26,321 - lava_callback - INFO - Received callback for node 69f2eb07e2fa3d7693ba90f8 INFO: 172.20.0.1:42652 - "POST /node/69f2eb07e2fa3d7693ba90f8 HTTP/1.0" 202 Accepted 2026-05-02 05:39:26,327 - lava_callback - INFO - Starting async job submit for node 69f2eb07e2fa3d7693ba90f8 2026-05-02 05:39:26,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:39:26,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eb07e2fa3d7693ba90f8 HTTP/1.1" 200 None 2026-05-02 05:39:26,342 - lava_callback - DEBUG - Processing node 69f2eb07e2fa3d7693ba90f8: baseline-arm64-baylibre 2026-05-02 05:39:26,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2eb07e2fa3d7693ba90f8 2026-05-02 05:39:26,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:39:26,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eb07e2fa3d7693ba90f8 HTTP/1.1" 200 None 2026-05-02 05:39:26,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:39:26,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 05:39:26,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:39:26,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2eb07e2fa3d7693ba90f8 HTTP/1.1" 200 None 2026-05-02 05:39:26,392 - lava_callback - ERROR - Error processing callback for node 69f2eb07e2fa3d7693ba90f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51588 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:40:46,523 - lava_callback - INFO - Received callback for node 69f2eb51e2fa3d7693ba912b INFO: 172.20.0.1:51590 - "POST /node/69f2eb51e2fa3d7693ba912b HTTP/1.0" 202 Accepted 2026-05-02 05:40:46,528 - lava_callback - INFO - Starting async job submit for node 69f2eb51e2fa3d7693ba912b 2026-05-02 05:40:46,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:40:46,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eb51e2fa3d7693ba912b HTTP/1.1" 200 None 2026-05-02 05:40:46,543 - lava_callback - DEBUG - Processing node 69f2eb51e2fa3d7693ba912b: baseline-arm64-baylibre 2026-05-02 05:40:46,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2eb51e2fa3d7693ba912b 2026-05-02 05:40:46,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:40:46,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eb51e2fa3d7693ba912b HTTP/1.1" 200 None 2026-05-02 05:40:46,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:40:46,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 05:40:46,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:40:46,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2eb51e2fa3d7693ba912b HTTP/1.1" 200 None 2026-05-02 05:40:46,594 - lava_callback - ERROR - Error processing callback for node 69f2eb51e2fa3d7693ba912b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45540 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:42:06,593 - lava_callback - INFO - Received callback for node 69f2eb9ae2fa3d7693ba914c INFO: 172.20.0.1:45548 - "POST /node/69f2eb9ae2fa3d7693ba914c HTTP/1.0" 202 Accepted 2026-05-02 05:42:06,598 - lava_callback - INFO - Starting async job submit for node 69f2eb9ae2fa3d7693ba914c 2026-05-02 05:42:06,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:42:06,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eb9ae2fa3d7693ba914c HTTP/1.1" 200 None 2026-05-02 05:42:06,613 - lava_callback - DEBUG - Processing node 69f2eb9ae2fa3d7693ba914c: baseline-arm64-baylibre 2026-05-02 05:42:06,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2eb9ae2fa3d7693ba914c 2026-05-02 05:42:06,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:42:06,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eb9ae2fa3d7693ba914c HTTP/1.1" 200 None 2026-05-02 05:42:06,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:42:06,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 05:42:06,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:42:06,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2eb9ae2fa3d7693ba914c HTTP/1.1" 200 None 2026-05-02 05:42:06,664 - lava_callback - ERROR - Error processing callback for node 69f2eb9ae2fa3d7693ba914c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37848 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:43:26,210 - lava_callback - INFO - Received callback for node 69f2ebeae2fa3d7693ba9183 INFO: 172.20.0.1:37854 - "POST /node/69f2ebeae2fa3d7693ba9183 HTTP/1.0" 202 Accepted 2026-05-02 05:43:26,216 - lava_callback - INFO - Starting async job submit for node 69f2ebeae2fa3d7693ba9183 2026-05-02 05:43:26,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:43:26,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ebeae2fa3d7693ba9183 HTTP/1.1" 200 None 2026-05-02 05:43:26,240 - lava_callback - DEBUG - Processing node 69f2ebeae2fa3d7693ba9183: baseline-arm64-clabbe 2026-05-02 05:43:26,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ebeae2fa3d7693ba9183 2026-05-02 05:43:26,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:43:26,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ebeae2fa3d7693ba9183 HTTP/1.1" 200 None 2026-05-02 05:43:26,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:43:26,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:43:26,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:43:26,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ebeae2fa3d7693ba9183 HTTP/1.1" 200 None 2026-05-02 05:43:26,285 - lava_callback - ERROR - Error processing callback for node 69f2ebeae2fa3d7693ba9183: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56466 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:44:26,668 - lava_callback - INFO - Received callback for node 69f2ec2ee2fa3d7693ba91a3 INFO: 172.20.0.1:56480 - "POST /node/69f2ec2ee2fa3d7693ba91a3 HTTP/1.0" 202 Accepted 2026-05-02 05:44:26,673 - lava_callback - INFO - Starting async job submit for node 69f2ec2ee2fa3d7693ba91a3 2026-05-02 05:44:26,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:44:26,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ec2ee2fa3d7693ba91a3 HTTP/1.1" 200 None 2026-05-02 05:44:26,689 - lava_callback - DEBUG - Processing node 69f2ec2ee2fa3d7693ba91a3: baseline-arm64-baylibre 2026-05-02 05:44:26,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ec2ee2fa3d7693ba91a3 2026-05-02 05:44:26,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:44:26,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ec2ee2fa3d7693ba91a3 HTTP/1.1" 200 None 2026-05-02 05:44:26,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:44:26,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 05:44:26,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:44:26,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ec2ee2fa3d7693ba91a3 HTTP/1.1" 200 None 2026-05-02 05:44:26,750 - lava_callback - ERROR - Error processing callback for node 69f2ec2ee2fa3d7693ba91a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51224 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:45:34,731 - lava_callback - INFO - Received callback for node 69f58a11ed4ed40b0224254a INFO: 172.20.0.1:46970 - "POST /node/69f58a11ed4ed40b0224254a HTTP/1.0" 202 Accepted 2026-05-02 05:45:34,738 - lava_callback - INFO - Starting async job submit for node 69f58a11ed4ed40b0224254a 2026-05-02 05:45:34,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:34,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58a11ed4ed40b0224254a HTTP/1.1" 200 None 2026-05-02 05:45:34,777 - lava_callback - DEBUG - Processing node 69f58a11ed4ed40b0224254a: baseline-nfs-arm64 2026-05-02 05:45:35,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:45:35,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:45:35,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f58a11ed4ed40b0224254a/log.txt.gz 2026-05-02 05:45:35,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:45:35,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:45:35,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f58a11ed4ed40b0224254a/lava-logs.yaml 2026-05-02 05:45:35,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-02 05:45:36,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-02 05:45:36,085 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f58a11ed4ed40b0224254a/lava_callback.json.gz 2026-05-02 05:45:36,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:36,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f58a11ed4ed40b0224254a HTTP/1.1" 200 None 2026-05-02 05:45:36,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:36,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5446ded4ed40b0223db13 HTTP/1.1" 200 None 2026-05-02 05:45:36,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:36,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f58a11ed4ed40b0224254a HTTP/1.1" 200 None 2026-05-02 05:45:36,229 - lava_callback - INFO - Completed processing callback for node 69f58a11ed4ed40b0224254a INFO: 172.20.0.1:46972 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:45:46,371 - lava_callback - INFO - Received callback for node 69f2ec7fe2fa3d7693ba91d4 INFO: 172.20.0.1:32770 - "POST /node/69f2ec7fe2fa3d7693ba91d4 HTTP/1.0" 202 Accepted 2026-05-02 05:45:46,377 - lava_callback - INFO - Starting async job submit for node 69f2ec7fe2fa3d7693ba91d4 2026-05-02 05:45:46,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:46,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ec7fe2fa3d7693ba91d4 HTTP/1.1" 200 None 2026-05-02 05:45:46,392 - lava_callback - DEBUG - Processing node 69f2ec7fe2fa3d7693ba91d4: baseline-arm64-clabbe 2026-05-02 05:45:46,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ec7fe2fa3d7693ba91d4 2026-05-02 05:45:46,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:46,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ec7fe2fa3d7693ba91d4 HTTP/1.1" 200 None 2026-05-02 05:45:46,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:46,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:45:46,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:46,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ec7fe2fa3d7693ba91d4 HTTP/1.1" 200 None 2026-05-02 05:45:46,446 - lava_callback - ERROR - Error processing callback for node 69f2ec7fe2fa3d7693ba91d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32772 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:45:53,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:45:53,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34584 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:46:46,795 - lava_callback - INFO - Received callback for node 69f2ecc1e2fa3d7693ba91f1 INFO: 172.20.0.1:34592 - "POST /node/69f2ecc1e2fa3d7693ba91f1 HTTP/1.0" 202 Accepted 2026-05-02 05:46:46,801 - lava_callback - INFO - Starting async job submit for node 69f2ecc1e2fa3d7693ba91f1 2026-05-02 05:46:46,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:46:46,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ecc1e2fa3d7693ba91f1 HTTP/1.1" 200 None 2026-05-02 05:46:46,818 - lava_callback - DEBUG - Processing node 69f2ecc1e2fa3d7693ba91f1: baseline-arm64-baylibre 2026-05-02 05:46:46,818 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ecc1e2fa3d7693ba91f1 2026-05-02 05:46:46,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:46:46,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ecc1e2fa3d7693ba91f1 HTTP/1.1" 200 None 2026-05-02 05:46:46,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:46:46,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 05:46:46,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:46:46,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ecc1e2fa3d7693ba91f1 HTTP/1.1" 200 None 2026-05-02 05:46:46,868 - lava_callback - ERROR - Error processing callback for node 69f2ecc1e2fa3d7693ba91f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51738 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:48:06,928 - lava_callback - INFO - Received callback for node 69f2ed0ee2fa3d7693ba920e INFO: 172.20.0.1:51742 - "POST /node/69f2ed0ee2fa3d7693ba920e HTTP/1.0" 202 Accepted 2026-05-02 05:48:06,934 - lava_callback - INFO - Starting async job submit for node 69f2ed0ee2fa3d7693ba920e 2026-05-02 05:48:06,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:48:06,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ed0ee2fa3d7693ba920e HTTP/1.1" 200 None 2026-05-02 05:48:06,950 - lava_callback - DEBUG - Processing node 69f2ed0ee2fa3d7693ba920e: baseline-arm64-baylibre 2026-05-02 05:48:06,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ed0ee2fa3d7693ba920e 2026-05-02 05:48:06,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:48:06,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ed0ee2fa3d7693ba920e HTTP/1.1" 200 None 2026-05-02 05:48:06,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:48:06,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 05:48:06,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:48:06,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ed0ee2fa3d7693ba920e HTTP/1.1" 200 None 2026-05-02 05:48:06,997 - lava_callback - ERROR - Error processing callback for node 69f2ed0ee2fa3d7693ba920e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:49:26,493 - lava_callback - INFO - Received callback for node 69f2ed5fe2fa3d7693ba9239 INFO: 172.20.0.1:58646 - "POST /node/69f2ed5fe2fa3d7693ba9239 HTTP/1.0" 202 Accepted 2026-05-02 05:49:26,503 - lava_callback - INFO - Starting async job submit for node 69f2ed5fe2fa3d7693ba9239 2026-05-02 05:49:26,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:58656 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:49:26,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ed5fe2fa3d7693ba9239 HTTP/1.1" 200 None 2026-05-02 05:49:26,519 - lava_callback - DEBUG - Processing node 69f2ed5fe2fa3d7693ba9239: baseline-arm64-clabbe 2026-05-02 05:49:26,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ed5fe2fa3d7693ba9239 2026-05-02 05:49:26,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:49:26,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ed5fe2fa3d7693ba9239 HTTP/1.1" 200 None 2026-05-02 05:49:26,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:49:26,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e108305534e97233243 HTTP/1.1" 200 None 2026-05-02 05:49:26,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:49:26,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ed5fe2fa3d7693ba9239 HTTP/1.1" 200 None 2026-05-02 05:49:26,577 - lava_callback - ERROR - Error processing callback for node 69f2ed5fe2fa3d7693ba9239: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:50:47,414 - lava_callback - INFO - Received callback for node 69f2eda5e2fa3d7693ba9256 INFO: 172.20.0.1:53900 - "POST /node/69f2eda5e2fa3d7693ba9256 HTTP/1.0" 202 Accepted 2026-05-02 05:50:47,420 - lava_callback - INFO - Starting async job submit for node 69f2eda5e2fa3d7693ba9256 2026-05-02 05:50:47,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:50:47,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eda5e2fa3d7693ba9256 HTTP/1.1" 200 None 2026-05-02 05:50:47,435 - lava_callback - DEBUG - Processing node 69f2eda5e2fa3d7693ba9256: baseline-arm64-baylibre 2026-05-02 05:50:47,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2eda5e2fa3d7693ba9256 2026-05-02 05:50:47,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:50:47,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eda5e2fa3d7693ba9256 HTTP/1.1" 200 None 2026-05-02 05:50:47,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:50:47,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 05:50:47,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:50:47,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2eda5e2fa3d7693ba9256 HTTP/1.1" 200 None 2026-05-02 05:50:47,487 - lava_callback - ERROR - Error processing callback for node 69f2eda5e2fa3d7693ba9256: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:51:47,052 - lava_callback - INFO - Received callback for node 69f2edefe2fa3d7693ba9281 INFO: 172.20.0.1:41196 - "POST /node/69f2edefe2fa3d7693ba9281 HTTP/1.0" 202 Accepted 2026-05-02 05:51:47,060 - lava_callback - INFO - Starting async job submit for node 69f2edefe2fa3d7693ba9281 2026-05-02 05:51:47,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:51:47,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2edefe2fa3d7693ba9281 HTTP/1.1" 200 None 2026-05-02 05:51:47,075 - lava_callback - DEBUG - Processing node 69f2edefe2fa3d7693ba9281: baseline-arm64-baylibre 2026-05-02 05:51:47,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2edefe2fa3d7693ba9281 2026-05-02 05:51:47,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:51:47,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2edefe2fa3d7693ba9281 HTTP/1.1" 200 None 2026-05-02 05:51:47,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:51:47,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 05:51:47,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:51:47,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2edefe2fa3d7693ba9281 HTTP/1.1" 200 None 2026-05-02 05:51:47,166 - lava_callback - ERROR - Error processing callback for node 69f2edefe2fa3d7693ba9281: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39370 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:53:07,141 - lava_callback - INFO - Received callback for node 69f2ee39e2fa3d7693ba929f INFO: 172.20.0.1:39386 - "POST /node/69f2ee39e2fa3d7693ba929f HTTP/1.0" 202 Accepted 2026-05-02 05:53:07,146 - lava_callback - INFO - Starting async job submit for node 69f2ee39e2fa3d7693ba929f 2026-05-02 05:53:07,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:53:07,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ee39e2fa3d7693ba929f HTTP/1.1" 200 None 2026-05-02 05:53:07,161 - lava_callback - DEBUG - Processing node 69f2ee39e2fa3d7693ba929f: baseline-arm64-baylibre 2026-05-02 05:53:07,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ee39e2fa3d7693ba929f 2026-05-02 05:53:07,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:53:07,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ee39e2fa3d7693ba929f HTTP/1.1" 200 None 2026-05-02 05:53:07,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:53:07,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 05:53:07,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:53:07,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ee39e2fa3d7693ba929f HTTP/1.1" 200 None 2026-05-02 05:53:07,207 - lava_callback - ERROR - Error processing callback for node 69f2ee39e2fa3d7693ba929f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44782 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:54:27,352 - lava_callback - INFO - Received callback for node 69f2ee83e2fa3d7693ba92cb INFO: 172.20.0.1:44792 - "POST /node/69f2ee83e2fa3d7693ba92cb HTTP/1.0" 202 Accepted 2026-05-02 05:54:27,357 - lava_callback - INFO - Starting async job submit for node 69f2ee83e2fa3d7693ba92cb 2026-05-02 05:54:27,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:54:27,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ee83e2fa3d7693ba92cb HTTP/1.1" 200 None 2026-05-02 05:54:27,372 - lava_callback - DEBUG - Processing node 69f2ee83e2fa3d7693ba92cb: baseline-arm64-baylibre 2026-05-02 05:54:27,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ee83e2fa3d7693ba92cb 2026-05-02 05:54:27,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:54:27,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ee83e2fa3d7693ba92cb HTTP/1.1" 200 None 2026-05-02 05:54:27,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:54:27,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 05:54:27,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:54:27,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ee83e2fa3d7693ba92cb HTTP/1.1" 200 None 2026-05-02 05:54:27,423 - lava_callback - ERROR - Error processing callback for node 69f2ee83e2fa3d7693ba92cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56412 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:55:27,224 - lava_callback - INFO - Received callback for node 69f2eec2e2fa3d7693ba92e7 INFO: 172.20.0.1:56416 - "POST /node/69f2eec2e2fa3d7693ba92e7 HTTP/1.0" 202 Accepted 2026-05-02 05:55:27,230 - lava_callback - INFO - Starting async job submit for node 69f2eec2e2fa3d7693ba92e7 2026-05-02 05:55:27,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:55:27,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eec2e2fa3d7693ba92e7 HTTP/1.1" 200 None 2026-05-02 05:55:27,245 - lava_callback - DEBUG - Processing node 69f2eec2e2fa3d7693ba92e7: baseline-arm-baylibre 2026-05-02 05:55:27,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2eec2e2fa3d7693ba92e7 2026-05-02 05:55:27,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:55:27,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eec2e2fa3d7693ba92e7 HTTP/1.1" 200 None 2026-05-02 05:55:27,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:55:27,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:55:27,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:55:27,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2eec2e2fa3d7693ba92e7 HTTP/1.1" 200 None 2026-05-02 05:55:27,303 - lava_callback - ERROR - Error processing callback for node 69f2eec2e2fa3d7693ba92e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42694 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:56:07,346 - lava_callback - INFO - Received callback for node 69f2eee7e2fa3d7693ba92f7 INFO: 172.20.0.1:42696 - "POST /node/69f2eee7e2fa3d7693ba92f7 HTTP/1.0" 202 Accepted 2026-05-02 05:56:07,352 - lava_callback - INFO - Starting async job submit for node 69f2eee7e2fa3d7693ba92f7 2026-05-02 05:56:07,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:07,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eee7e2fa3d7693ba92f7 HTTP/1.1" 200 None 2026-05-02 05:56:07,367 - lava_callback - DEBUG - Processing node 69f2eee7e2fa3d7693ba92f7: baseline-arm-baylibre 2026-05-02 05:56:07,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2eee7e2fa3d7693ba92f7 2026-05-02 05:56:07,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:07,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2eee7e2fa3d7693ba92f7 HTTP/1.1" 200 None 2026-05-02 05:56:07,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:07,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:56:07,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:07,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2eee7e2fa3d7693ba92f7 HTTP/1.1" 200 None 2026-05-02 05:56:07,423 - lava_callback - ERROR - Error processing callback for node 69f2eee7e2fa3d7693ba92f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41696 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:56:47,281 - lava_callback - INFO - Received callback for node 69f2ef0ce2fa3d7693ba9300 INFO: 172.20.0.1:41700 - "POST /node/69f2ef0ce2fa3d7693ba9300 HTTP/1.0" 202 Accepted 2026-05-02 05:56:47,287 - lava_callback - INFO - Starting async job submit for node 69f2ef0ce2fa3d7693ba9300 2026-05-02 05:56:47,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef0ce2fa3d7693ba9300 HTTP/1.1" 200 None 2026-05-02 05:56:47,308 - lava_callback - DEBUG - Processing node 69f2ef0ce2fa3d7693ba9300: baseline-arm-baylibre 2026-05-02 05:56:47,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ef0ce2fa3d7693ba9300 2026-05-02 05:56:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:47,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef0ce2fa3d7693ba9300 HTTP/1.1" 200 None 2026-05-02 05:56:47,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:47,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:56:47,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:56:47,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ef0ce2fa3d7693ba9300 HTTP/1.1" 200 None 2026-05-02 05:56:47,358 - lava_callback - ERROR - Error processing callback for node 69f2ef0ce2fa3d7693ba9300: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45222 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:57:07,276 - lava_callback - INFO - Received callback for node 69f2ef31e2fa3d7693ba930d INFO: 172.20.0.1:45234 - "POST /node/69f2ef31e2fa3d7693ba930d HTTP/1.0" 202 Accepted 2026-05-02 05:57:07,282 - lava_callback - INFO - Starting async job submit for node 69f2ef31e2fa3d7693ba930d 2026-05-02 05:57:07,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:07,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef31e2fa3d7693ba930d HTTP/1.1" 200 None 2026-05-02 05:57:07,298 - lava_callback - DEBUG - Processing node 69f2ef31e2fa3d7693ba930d: baseline-arm-baylibre 2026-05-02 05:57:07,298 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ef31e2fa3d7693ba930d 2026-05-02 05:57:07,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:07,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef31e2fa3d7693ba930d HTTP/1.1" 200 None 2026-05-02 05:57:07,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:07,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:57:07,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:07,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ef31e2fa3d7693ba930d HTTP/1.1" 200 None 2026-05-02 05:57:07,354 - lava_callback - ERROR - Error processing callback for node 69f2ef31e2fa3d7693ba930d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53782 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:57:47,356 - lava_callback - INFO - Received callback for node 69f2ef56e2fa3d7693ba9318 INFO: 172.20.0.1:53796 - "POST /node/69f2ef56e2fa3d7693ba9318 HTTP/1.0" 202 Accepted 2026-05-02 05:57:47,361 - lava_callback - INFO - Starting async job submit for node 69f2ef56e2fa3d7693ba9318 2026-05-02 05:57:47,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:47,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef56e2fa3d7693ba9318 HTTP/1.1" 200 None 2026-05-02 05:57:47,391 - lava_callback - DEBUG - Processing node 69f2ef56e2fa3d7693ba9318: baseline-arm-baylibre 2026-05-02 05:57:47,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ef56e2fa3d7693ba9318 2026-05-02 05:57:47,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:47,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef56e2fa3d7693ba9318 HTTP/1.1" 200 None 2026-05-02 05:57:47,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:47,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:57:47,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:57:47,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ef56e2fa3d7693ba9318 HTTP/1.1" 200 None 2026-05-02 05:57:47,440 - lava_callback - ERROR - Error processing callback for node 69f2ef56e2fa3d7693ba9318: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34908 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:58:27,358 - lava_callback - INFO - Received callback for node 69f2ef7be2fa3d7693ba9323 INFO: 172.20.0.1:34914 - "POST /node/69f2ef7be2fa3d7693ba9323 HTTP/1.0" 202 Accepted 2026-05-02 05:58:27,364 - lava_callback - INFO - Starting async job submit for node 69f2ef7be2fa3d7693ba9323 2026-05-02 05:58:27,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:58:27,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef7be2fa3d7693ba9323 HTTP/1.1" 200 None 2026-05-02 05:58:27,388 - lava_callback - DEBUG - Processing node 69f2ef7be2fa3d7693ba9323: baseline-arm-baylibre 2026-05-02 05:58:27,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ef7be2fa3d7693ba9323 2026-05-02 05:58:27,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:58:27,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ef7be2fa3d7693ba9323 HTTP/1.1" 200 None 2026-05-02 05:58:27,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:58:27,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 05:58:27,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:58:27,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ef7be2fa3d7693ba9323 HTTP/1.1" 200 None 2026-05-02 05:58:27,435 - lava_callback - ERROR - Error processing callback for node 69f2ef7be2fa3d7693ba9323: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34428 - "GET / HTTP/1.0" 200 OK 2026-05-02 05:59:26,853 - lava_callback - INFO - Received callback for node 69f2efafe2fa3d7693ba9333 INFO: 172.20.0.1:34444 - "POST /node/69f2efafe2fa3d7693ba9333 HTTP/1.0" 202 Accepted 2026-05-02 05:59:26,859 - lava_callback - INFO - Starting async job submit for node 69f2efafe2fa3d7693ba9333 2026-05-02 05:59:26,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:59:26,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2efafe2fa3d7693ba9333 HTTP/1.1" 200 None 2026-05-02 05:59:26,874 - lava_callback - DEBUG - Processing node 69f2efafe2fa3d7693ba9333: baseline-arm64-clabbe 2026-05-02 05:59:26,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2efafe2fa3d7693ba9333 2026-05-02 05:59:26,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:59:26,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2efafe2fa3d7693ba9333 HTTP/1.1" 200 None 2026-05-02 05:59:26,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:59:26,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 05:59:26,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 05:59:26,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2efafe2fa3d7693ba9333 HTTP/1.1" 200 None 2026-05-02 05:59:26,928 - lava_callback - ERROR - Error processing callback for node 69f2efafe2fa3d7693ba9333: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38296 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:00:26,962 - lava_callback - INFO - Received callback for node 69f2efefe2fa3d7693ba934f INFO: 172.20.0.1:38308 - "POST /node/69f2efefe2fa3d7693ba934f HTTP/1.0" 202 Accepted 2026-05-02 06:00:26,967 - lava_callback - INFO - Starting async job submit for node 69f2efefe2fa3d7693ba934f 2026-05-02 06:00:26,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:00:27,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2efefe2fa3d7693ba934f HTTP/1.1" 200 None 2026-05-02 06:00:27,009 - lava_callback - DEBUG - Processing node 69f2efefe2fa3d7693ba934f: baseline-arm-clabbe 2026-05-02 06:00:27,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2efefe2fa3d7693ba934f 2026-05-02 06:00:27,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:00:27,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2efefe2fa3d7693ba934f HTTP/1.1" 200 None 2026-05-02 06:00:27,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:00:27,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:00:27,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:00:27,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2efefe2fa3d7693ba934f HTTP/1.1" 200 None 2026-05-02 06:00:27,067 - lava_callback - ERROR - Error processing callback for node 69f2efefe2fa3d7693ba934f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:01:27,206 - lava_callback - INFO - Received callback for node 69f2f022e2fa3d7693ba9365 INFO: 172.20.0.1:51762 - "POST /node/69f2f022e2fa3d7693ba9365 HTTP/1.0" 202 Accepted 2026-05-02 06:01:27,211 - lava_callback - INFO - Starting async job submit for node 69f2f022e2fa3d7693ba9365 2026-05-02 06:01:27,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:01:27,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f022e2fa3d7693ba9365 HTTP/1.1" 200 None 2026-05-02 06:01:27,227 - lava_callback - DEBUG - Processing node 69f2f022e2fa3d7693ba9365: baseline-arm64-clabbe 2026-05-02 06:01:27,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f022e2fa3d7693ba9365 2026-05-02 06:01:27,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:01:27,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f022e2fa3d7693ba9365 HTTP/1.1" 200 None 2026-05-02 06:01:27,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:01:27,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:01:27,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:01:27,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f022e2fa3d7693ba9365 HTTP/1.1" 200 None 2026-05-02 06:01:27,284 - lava_callback - ERROR - Error processing callback for node 69f2f022e2fa3d7693ba9365: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52920 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:02:27,130 - lava_callback - INFO - Received callback for node 69f2f060e2fa3d7693ba9381 INFO: 172.20.0.1:52926 - "POST /node/69f2f060e2fa3d7693ba9381 HTTP/1.0" 202 Accepted 2026-05-02 06:02:27,135 - lava_callback - INFO - Starting async job submit for node 69f2f060e2fa3d7693ba9381 2026-05-02 06:02:27,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:27,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f060e2fa3d7693ba9381 HTTP/1.1" 200 None 2026-05-02 06:02:27,156 - lava_callback - DEBUG - Processing node 69f2f060e2fa3d7693ba9381: baseline-arm-clabbe 2026-05-02 06:02:27,156 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f060e2fa3d7693ba9381 2026-05-02 06:02:27,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:27,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f060e2fa3d7693ba9381 HTTP/1.1" 200 None 2026-05-02 06:02:27,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:27,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:02:27,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:27,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f060e2fa3d7693ba9381 HTTP/1.1" 200 None 2026-05-02 06:02:27,201 - lava_callback - ERROR - Error processing callback for node 69f2f060e2fa3d7693ba9381: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:02:47,052 - lava_callback - INFO - Received callback for node 69f2f084e2fa3d7693ba938c INFO: 172.20.0.1:53934 - "POST /node/69f2f084e2fa3d7693ba938c HTTP/1.0" 202 Accepted 2026-05-02 06:02:47,058 - lava_callback - INFO - Starting async job submit for node 69f2f084e2fa3d7693ba938c 2026-05-02 06:02:47,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:47,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f084e2fa3d7693ba938c HTTP/1.1" 200 None 2026-05-02 06:02:47,074 - lava_callback - DEBUG - Processing node 69f2f084e2fa3d7693ba938c: baseline-arm-clabbe 2026-05-02 06:02:47,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f084e2fa3d7693ba938c 2026-05-02 06:02:47,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:47,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f084e2fa3d7693ba938c HTTP/1.1" 200 None 2026-05-02 06:02:47,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:47,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:02:47,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:02:47,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f084e2fa3d7693ba938c HTTP/1.1" 200 None 2026-05-02 06:02:47,126 - lava_callback - ERROR - Error processing callback for node 69f2f084e2fa3d7693ba938c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:03:47,105 - lava_callback - INFO - Received callback for node 69f2f0b6e2fa3d7693ba93a0 INFO: 172.20.0.1:46894 - "POST /node/69f2f0b6e2fa3d7693ba93a0 HTTP/1.0" 202 Accepted 2026-05-02 06:03:47,111 - lava_callback - INFO - Starting async job submit for node 69f2f0b6e2fa3d7693ba93a0 2026-05-02 06:03:47,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:03:47,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f0b6e2fa3d7693ba93a0 HTTP/1.1" 200 None 2026-05-02 06:03:47,126 - lava_callback - DEBUG - Processing node 69f2f0b6e2fa3d7693ba93a0: baseline-arm64-clabbe 2026-05-02 06:03:47,126 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f0b6e2fa3d7693ba93a0 2026-05-02 06:03:47,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:03:47,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f0b6e2fa3d7693ba93a0 HTTP/1.1" 200 None 2026-05-02 06:03:47,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:03:47,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:03:47,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:03:47,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f0b6e2fa3d7693ba93a0 HTTP/1.1" 200 None 2026-05-02 06:03:47,179 - lava_callback - ERROR - Error processing callback for node 69f2f0b6e2fa3d7693ba93a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34862 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:04:47,150 - lava_callback - INFO - Received callback for node 69f2f0f6e2fa3d7693ba93bb INFO: 172.20.0.1:34864 - "POST /node/69f2f0f6e2fa3d7693ba93bb HTTP/1.0" 202 Accepted 2026-05-02 06:04:47,155 - lava_callback - INFO - Starting async job submit for node 69f2f0f6e2fa3d7693ba93bb 2026-05-02 06:04:47,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:04:47,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f0f6e2fa3d7693ba93bb HTTP/1.1" 200 None 2026-05-02 06:04:47,171 - lava_callback - DEBUG - Processing node 69f2f0f6e2fa3d7693ba93bb: baseline-arm-clabbe 2026-05-02 06:04:47,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f0f6e2fa3d7693ba93bb 2026-05-02 06:04:47,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:04:47,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f0f6e2fa3d7693ba93bb HTTP/1.1" 200 None 2026-05-02 06:04:47,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:04:47,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:04:47,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:04:47,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f0f6e2fa3d7693ba93bb HTTP/1.1" 200 None 2026-05-02 06:04:47,222 - lava_callback - ERROR - Error processing callback for node 69f2f0f6e2fa3d7693ba93bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:05:27,153 - lava_callback - INFO - Received callback for node 69f2f11de2fa3d7693ba93c6 INFO: 172.20.0.1:51414 - "POST /node/69f2f11de2fa3d7693ba93c6 HTTP/1.0" 202 Accepted 2026-05-02 06:05:27,159 - lava_callback - INFO - Starting async job submit for node 69f2f11de2fa3d7693ba93c6 2026-05-02 06:05:27,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:05:27,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f11de2fa3d7693ba93c6 HTTP/1.1" 200 None 2026-05-02 06:05:27,173 - lava_callback - DEBUG - Processing node 69f2f11de2fa3d7693ba93c6: baseline-arm-clabbe 2026-05-02 06:05:27,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f11de2fa3d7693ba93c6 2026-05-02 06:05:27,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:05:27,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f11de2fa3d7693ba93c6 HTTP/1.1" 200 None 2026-05-02 06:05:27,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:05:27,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:05:27,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:05:27,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f11de2fa3d7693ba93c6 HTTP/1.1" 200 None 2026-05-02 06:05:27,218 - lava_callback - ERROR - Error processing callback for node 69f2f11de2fa3d7693ba93c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35522 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:06:07,214 - lava_callback - INFO - Received callback for node 69f2f143e2fa3d7693ba93d1 INFO: 172.20.0.1:35536 - "POST /node/69f2f143e2fa3d7693ba93d1 HTTP/1.0" 202 Accepted 2026-05-02 06:06:07,219 - lava_callback - INFO - Starting async job submit for node 69f2f143e2fa3d7693ba93d1 2026-05-02 06:06:07,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:06:07,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f143e2fa3d7693ba93d1 HTTP/1.1" 200 None 2026-05-02 06:06:07,237 - lava_callback - DEBUG - Processing node 69f2f143e2fa3d7693ba93d1: baseline-arm-clabbe 2026-05-02 06:06:07,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f143e2fa3d7693ba93d1 2026-05-02 06:06:07,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:06:07,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f143e2fa3d7693ba93d1 HTTP/1.1" 200 None 2026-05-02 06:06:07,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:06:07,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:06:07,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:06:07,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f143e2fa3d7693ba93d1 HTTP/1.1" 200 None 2026-05-02 06:06:07,282 - lava_callback - ERROR - Error processing callback for node 69f2f143e2fa3d7693ba93d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56498 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:07:07,250 - lava_callback - INFO - Received callback for node 69f2f177e2fa3d7693ba93de INFO: 172.20.0.1:56508 - "POST /node/69f2f177e2fa3d7693ba93de HTTP/1.0" 202 Accepted 2026-05-02 06:07:07,255 - lava_callback - INFO - Starting async job submit for node 69f2f177e2fa3d7693ba93de 2026-05-02 06:07:07,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:07:07,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f177e2fa3d7693ba93de HTTP/1.1" 200 None 2026-05-02 06:07:07,270 - lava_callback - DEBUG - Processing node 69f2f177e2fa3d7693ba93de: baseline-arm64-clabbe 2026-05-02 06:07:07,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f177e2fa3d7693ba93de 2026-05-02 06:07:07,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:07:07,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f177e2fa3d7693ba93de HTTP/1.1" 200 None 2026-05-02 06:07:07,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:07:07,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:07:07,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:07:07,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f177e2fa3d7693ba93de HTTP/1.1" 200 None 2026-05-02 06:07:07,316 - lava_callback - ERROR - Error processing callback for node 69f2f177e2fa3d7693ba93de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:08:07,312 - lava_callback - INFO - Received callback for node 69f2f1b5e2fa3d7693ba93fb INFO: 172.20.0.1:36938 - "POST /node/69f2f1b5e2fa3d7693ba93fb HTTP/1.0" 202 Accepted 2026-05-02 06:08:07,318 - lava_callback - INFO - Starting async job submit for node 69f2f1b5e2fa3d7693ba93fb 2026-05-02 06:08:07,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:07,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f1b5e2fa3d7693ba93fb HTTP/1.1" 200 None 2026-05-02 06:08:07,333 - lava_callback - DEBUG - Processing node 69f2f1b5e2fa3d7693ba93fb: baseline-arm-clabbe 2026-05-02 06:08:07,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f1b5e2fa3d7693ba93fb 2026-05-02 06:08:07,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:07,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f1b5e2fa3d7693ba93fb HTTP/1.1" 200 None 2026-05-02 06:08:07,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:07,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:08:07,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:07,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f1b5e2fa3d7693ba93fb HTTP/1.1" 200 None 2026-05-02 06:08:07,381 - lava_callback - ERROR - Error processing callback for node 69f2f1b5e2fa3d7693ba93fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60080 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:08:47,296 - lava_callback - INFO - Received callback for node 69f2f1dae2fa3d7693ba940b INFO: 172.20.0.1:60096 - "POST /node/69f2f1dae2fa3d7693ba940b HTTP/1.0" 202 Accepted 2026-05-02 06:08:47,302 - lava_callback - INFO - Starting async job submit for node 69f2f1dae2fa3d7693ba940b 2026-05-02 06:08:47,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:47,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f1dae2fa3d7693ba940b HTTP/1.1" 200 None 2026-05-02 06:08:47,317 - lava_callback - DEBUG - Processing node 69f2f1dae2fa3d7693ba940b: baseline-arm-clabbe 2026-05-02 06:08:47,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f1dae2fa3d7693ba940b 2026-05-02 06:08:47,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:47,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f1dae2fa3d7693ba940b HTTP/1.1" 200 None 2026-05-02 06:08:47,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:47,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:08:47,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:08:47,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f1dae2fa3d7693ba940b HTTP/1.1" 200 None 2026-05-02 06:08:47,364 - lava_callback - ERROR - Error processing callback for node 69f2f1dae2fa3d7693ba940b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33688 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:09:27,410 - lava_callback - INFO - Received callback for node 69f2f20de2fa3d7693ba941c INFO: 172.20.0.1:33690 - "POST /node/69f2f20de2fa3d7693ba941c HTTP/1.0" 202 Accepted 2026-05-02 06:09:27,416 - lava_callback - INFO - Starting async job submit for node 69f2f20de2fa3d7693ba941c 2026-05-02 06:09:27,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:09:27,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f20de2fa3d7693ba941c HTTP/1.1" 200 None 2026-05-02 06:09:27,431 - lava_callback - DEBUG - Processing node 69f2f20de2fa3d7693ba941c: baseline-arm64-clabbe 2026-05-02 06:09:27,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f20de2fa3d7693ba941c 2026-05-02 06:09:27,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:09:27,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f20de2fa3d7693ba941c HTTP/1.1" 200 None 2026-05-02 06:09:27,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:09:27,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:09:27,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:09:27,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f20de2fa3d7693ba941c HTTP/1.1" 200 None 2026-05-02 06:09:27,497 - lava_callback - ERROR - Error processing callback for node 69f2f20de2fa3d7693ba941c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:10:27,407 - lava_callback - INFO - Received callback for node 69f2f24be2fa3d7693ba9438 INFO: 172.20.0.1:56022 - "POST /node/69f2f24be2fa3d7693ba9438 HTTP/1.0" 202 Accepted 2026-05-02 06:10:27,413 - lava_callback - INFO - Starting async job submit for node 69f2f24be2fa3d7693ba9438 2026-05-02 06:10:27,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:10:27,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f24be2fa3d7693ba9438 HTTP/1.1" 200 None 2026-05-02 06:10:27,429 - lava_callback - DEBUG - Processing node 69f2f24be2fa3d7693ba9438: baseline-arm-clabbe 2026-05-02 06:10:27,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f24be2fa3d7693ba9438 2026-05-02 06:10:27,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:10:27,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f24be2fa3d7693ba9438 HTTP/1.1" 200 None 2026-05-02 06:10:27,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:10:27,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5ea28305534e972332fe HTTP/1.1" 200 None 2026-05-02 06:10:27,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:10:27,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f24be2fa3d7693ba9438 HTTP/1.1" 200 None 2026-05-02 06:10:27,495 - lava_callback - ERROR - Error processing callback for node 69f2f24be2fa3d7693ba9438: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37712 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:11:27,464 - lava_callback - INFO - Received callback for node 69f2f282e2fa3d7693ba944b INFO: 172.20.0.1:37722 - "POST /node/69f2f282e2fa3d7693ba944b HTTP/1.0" 202 Accepted 2026-05-02 06:11:27,469 - lava_callback - INFO - Starting async job submit for node 69f2f282e2fa3d7693ba944b 2026-05-02 06:11:27,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:11:27,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f282e2fa3d7693ba944b HTTP/1.1" 200 None 2026-05-02 06:11:27,489 - lava_callback - DEBUG - Processing node 69f2f282e2fa3d7693ba944b: baseline-arm64-clabbe 2026-05-02 06:11:27,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f282e2fa3d7693ba944b 2026-05-02 06:11:27,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:11:27,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f282e2fa3d7693ba944b HTTP/1.1" 200 None 2026-05-02 06:11:27,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:11:27,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:11:27,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:11:27,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f282e2fa3d7693ba944b HTTP/1.1" 200 None 2026-05-02 06:11:27,534 - lava_callback - ERROR - Error processing callback for node 69f2f282e2fa3d7693ba944b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46678 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:12:28,197 - lava_callback - INFO - Received callback for node 69f2f2c6e2fa3d7693ba9467 INFO: 172.20.0.1:46690 - "POST /node/69f2f2c6e2fa3d7693ba9467 HTTP/1.0" 202 Accepted 2026-05-02 06:12:28,203 - lava_callback - INFO - Starting async job submit for node 69f2f2c6e2fa3d7693ba9467 2026-05-02 06:12:28,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:12:28,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f2c6e2fa3d7693ba9467 HTTP/1.1" 200 None 2026-05-02 06:12:28,220 - lava_callback - DEBUG - Processing node 69f2f2c6e2fa3d7693ba9467: baseline-arm64-baylibre 2026-05-02 06:12:28,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f2c6e2fa3d7693ba9467 2026-05-02 06:12:28,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:12:28,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f2c6e2fa3d7693ba9467 HTTP/1.1" 200 None 2026-05-02 06:12:28,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:12:28,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:12:28,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:12:28,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f2c6e2fa3d7693ba9467 HTTP/1.1" 200 None 2026-05-02 06:12:28,273 - lava_callback - ERROR - Error processing callback for node 69f2f2c6e2fa3d7693ba9467: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57648 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:13:48,127 - lava_callback - INFO - Received callback for node 69f2f311e2fa3d7693ba9493 INFO: 172.20.0.1:57652 - "POST /node/69f2f311e2fa3d7693ba9493 HTTP/1.0" 202 Accepted 2026-05-02 06:13:48,133 - lava_callback - INFO - Starting async job submit for node 69f2f311e2fa3d7693ba9493 2026-05-02 06:13:48,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:13:48,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f311e2fa3d7693ba9493 HTTP/1.1" 200 None 2026-05-02 06:13:48,155 - lava_callback - DEBUG - Processing node 69f2f311e2fa3d7693ba9493: baseline-arm64-baylibre 2026-05-02 06:13:48,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f311e2fa3d7693ba9493 2026-05-02 06:13:48,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:13:48,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f311e2fa3d7693ba9493 HTTP/1.1" 200 None 2026-05-02 06:13:48,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:13:48,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:13:48,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:13:48,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f311e2fa3d7693ba9493 HTTP/1.1" 200 None 2026-05-02 06:13:48,202 - lava_callback - ERROR - Error processing callback for node 69f2f311e2fa3d7693ba9493: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40018 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:15:07,515 - lava_callback - INFO - Received callback for node 69f2f362e2fa3d7693ba94b1 INFO: 172.20.0.1:40032 - "POST /node/69f2f362e2fa3d7693ba94b1 HTTP/1.0" 202 Accepted 2026-05-02 06:15:07,520 - lava_callback - INFO - Starting async job submit for node 69f2f362e2fa3d7693ba94b1 2026-05-02 06:15:07,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:15:07,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f362e2fa3d7693ba94b1 HTTP/1.1" 200 None 2026-05-02 06:15:07,535 - lava_callback - DEBUG - Processing node 69f2f362e2fa3d7693ba94b1: baseline-arm64-clabbe 2026-05-02 06:15:07,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f362e2fa3d7693ba94b1 2026-05-02 06:15:07,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:15:07,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f362e2fa3d7693ba94b1 HTTP/1.1" 200 None 2026-05-02 06:15:07,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:15:07,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:15:07,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:15:07,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f362e2fa3d7693ba94b1 HTTP/1.1" 200 None 2026-05-02 06:15:07,584 - lava_callback - ERROR - Error processing callback for node 69f2f362e2fa3d7693ba94b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38808 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:16:08,301 - lava_callback - INFO - Received callback for node 69f2f3a6e2fa3d7693ba94db INFO: 172.20.0.1:38818 - "POST /node/69f2f3a6e2fa3d7693ba94db HTTP/1.0" 202 Accepted 2026-05-02 06:16:08,306 - lava_callback - INFO - Starting async job submit for node 69f2f3a6e2fa3d7693ba94db 2026-05-02 06:16:08,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:16:08,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f3a6e2fa3d7693ba94db HTTP/1.1" 200 None 2026-05-02 06:16:08,323 - lava_callback - DEBUG - Processing node 69f2f3a6e2fa3d7693ba94db: baseline-arm64-baylibre 2026-05-02 06:16:08,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f3a6e2fa3d7693ba94db 2026-05-02 06:16:08,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:16:08,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f3a6e2fa3d7693ba94db HTTP/1.1" 200 None 2026-05-02 06:16:08,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:16:08,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:16:08,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:16:08,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f3a6e2fa3d7693ba94db HTTP/1.1" 200 None 2026-05-02 06:16:08,375 - lava_callback - ERROR - Error processing callback for node 69f2f3a6e2fa3d7693ba94db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:17:28,395 - lava_callback - INFO - Received callback for node 69f2f3f2e2fa3d7693ba94f9 INFO: 172.20.0.1:51396 - "POST /node/69f2f3f2e2fa3d7693ba94f9 HTTP/1.0" 202 Accepted 2026-05-02 06:17:28,401 - lava_callback - INFO - Starting async job submit for node 69f2f3f2e2fa3d7693ba94f9 2026-05-02 06:17:28,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:17:28,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f3f2e2fa3d7693ba94f9 HTTP/1.1" 200 None 2026-05-02 06:17:28,415 - lava_callback - DEBUG - Processing node 69f2f3f2e2fa3d7693ba94f9: baseline-arm64-baylibre 2026-05-02 06:17:28,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f3f2e2fa3d7693ba94f9 2026-05-02 06:17:28,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:17:28,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f3f2e2fa3d7693ba94f9 HTTP/1.1" 200 None 2026-05-02 06:17:28,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:17:28,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:17:28,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:17:28,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f3f2e2fa3d7693ba94f9 HTTP/1.1" 200 None 2026-05-02 06:17:28,460 - lava_callback - ERROR - Error processing callback for node 69f2f3f2e2fa3d7693ba94f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:18:48,452 - lava_callback - INFO - Received callback for node 69f2f43de2fa3d7693ba9524 INFO: 172.20.0.1:34656 - "POST /node/69f2f43de2fa3d7693ba9524 HTTP/1.0" 202 Accepted 2026-05-02 06:18:48,458 - lava_callback - INFO - Starting async job submit for node 69f2f43de2fa3d7693ba9524 2026-05-02 06:18:48,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:18:48,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f43de2fa3d7693ba9524 HTTP/1.1" 200 None 2026-05-02 06:18:48,472 - lava_callback - DEBUG - Processing node 69f2f43de2fa3d7693ba9524: baseline-arm64-baylibre 2026-05-02 06:18:48,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f43de2fa3d7693ba9524 2026-05-02 06:18:48,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:18:48,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f43de2fa3d7693ba9524 HTTP/1.1" 200 None 2026-05-02 06:18:48,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:18:48,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:18:48,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:18:48,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f43de2fa3d7693ba9524 HTTP/1.1" 200 None 2026-05-02 06:18:48,518 - lava_callback - ERROR - Error processing callback for node 69f2f43de2fa3d7693ba9524: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50168 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:20:08,439 - lava_callback - INFO - Received callback for node 69f2f488e2fa3d7693ba9548 INFO: 172.20.0.1:50176 - "POST /node/69f2f488e2fa3d7693ba9548 HTTP/1.0" 202 Accepted 2026-05-02 06:20:08,444 - lava_callback - INFO - Starting async job submit for node 69f2f488e2fa3d7693ba9548 2026-05-02 06:20:08,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:20:08,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f488e2fa3d7693ba9548 HTTP/1.1" 200 None 2026-05-02 06:20:08,462 - lava_callback - DEBUG - Processing node 69f2f488e2fa3d7693ba9548: baseline-arm64-baylibre 2026-05-02 06:20:08,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f488e2fa3d7693ba9548 2026-05-02 06:20:08,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:20:08,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f488e2fa3d7693ba9548 HTTP/1.1" 200 None 2026-05-02 06:20:08,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:20:08,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:20:08,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:20:08,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f488e2fa3d7693ba9548 HTTP/1.1" 200 None 2026-05-02 06:20:08,513 - lava_callback - ERROR - Error processing callback for node 69f2f488e2fa3d7693ba9548: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:21:27,894 - lava_callback - INFO - Received callback for node 69f2f4dae2fa3d7693ba9573 INFO: 172.20.0.1:51106 - "POST /node/69f2f4dae2fa3d7693ba9573 HTTP/1.0" 202 Accepted 2026-05-02 06:21:27,900 - lava_callback - INFO - Starting async job submit for node 69f2f4dae2fa3d7693ba9573 2026-05-02 06:21:27,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:21:27,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f4dae2fa3d7693ba9573 HTTP/1.1" 200 None 2026-05-02 06:21:27,915 - lava_callback - DEBUG - Processing node 69f2f4dae2fa3d7693ba9573: baseline-arm64-clabbe 2026-05-02 06:21:27,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f4dae2fa3d7693ba9573 2026-05-02 06:21:27,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:21:27,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f4dae2fa3d7693ba9573 HTTP/1.1" 200 None 2026-05-02 06:21:27,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:21:27,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:21:27,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:21:27,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f4dae2fa3d7693ba9573 HTTP/1.1" 200 None 2026-05-02 06:21:27,961 - lava_callback - ERROR - Error processing callback for node 69f2f4dae2fa3d7693ba9573: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35448 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:22:32,624 - lava_callback - INFO - Received callback for node 69f2f516e2fa3d7693ba958f INFO: 172.20.0.1:56090 - "POST /node/69f2f516e2fa3d7693ba958f HTTP/1.0" 202 Accepted 2026-05-02 06:22:32,629 - lava_callback - INFO - Starting async job submit for node 69f2f516e2fa3d7693ba958f 2026-05-02 06:22:32,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:22:32,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f516e2fa3d7693ba958f HTTP/1.1" 200 None 2026-05-02 06:22:32,644 - lava_callback - DEBUG - Processing node 69f2f516e2fa3d7693ba958f: baseline-arm-baylibre 2026-05-02 06:22:32,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f516e2fa3d7693ba958f 2026-05-02 06:22:32,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:22:32,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f516e2fa3d7693ba958f HTTP/1.1" 200 None 2026-05-02 06:22:32,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:22:32,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:22:32,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:22:32,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f516e2fa3d7693ba958f HTTP/1.1" 200 None 2026-05-02 06:22:32,715 - lava_callback - ERROR - Error processing callback for node 69f2f516e2fa3d7693ba958f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35762 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:23:08,666 - lava_callback - INFO - Received callback for node 69f2f53be2fa3d7693ba959f INFO: 172.20.0.1:35764 - "POST /node/69f2f53be2fa3d7693ba959f HTTP/1.0" 202 Accepted 2026-05-02 06:23:08,671 - lava_callback - INFO - Starting async job submit for node 69f2f53be2fa3d7693ba959f 2026-05-02 06:23:08,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:08,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f53be2fa3d7693ba959f HTTP/1.1" 200 None 2026-05-02 06:23:08,709 - lava_callback - DEBUG - Processing node 69f2f53be2fa3d7693ba959f: baseline-arm-baylibre 2026-05-02 06:23:08,709 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f53be2fa3d7693ba959f 2026-05-02 06:23:08,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:08,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f53be2fa3d7693ba959f HTTP/1.1" 200 None 2026-05-02 06:23:08,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:08,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:23:08,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:08,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f53be2fa3d7693ba959f HTTP/1.1" 200 None 2026-05-02 06:23:08,755 - lava_callback - ERROR - Error processing callback for node 69f2f53be2fa3d7693ba959f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60146 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:23:52,699 - lava_callback - INFO - Received callback for node 69f2f563e2fa3d7693ba95ac INFO: 172.20.0.1:47386 - "POST /node/69f2f563e2fa3d7693ba95ac HTTP/1.0" 202 Accepted 2026-05-02 06:23:52,705 - lava_callback - INFO - Starting async job submit for node 69f2f563e2fa3d7693ba95ac 2026-05-02 06:23:52,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:52,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f563e2fa3d7693ba95ac HTTP/1.1" 200 None 2026-05-02 06:23:52,720 - lava_callback - DEBUG - Processing node 69f2f563e2fa3d7693ba95ac: baseline-arm-baylibre 2026-05-02 06:23:52,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f563e2fa3d7693ba95ac 2026-05-02 06:23:52,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:52,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f563e2fa3d7693ba95ac HTTP/1.1" 200 None 2026-05-02 06:23:52,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:52,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:23:52,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:23:52,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f563e2fa3d7693ba95ac HTTP/1.1" 200 None 2026-05-02 06:23:52,769 - lava_callback - ERROR - Error processing callback for node 69f2f563e2fa3d7693ba95ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34914 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:24:48,054 - lava_callback - INFO - Received callback for node 69f2f59be2fa3d7693ba95b5 INFO: 172.20.0.1:34916 - "POST /node/69f2f59be2fa3d7693ba95b5 HTTP/1.0" 202 Accepted 2026-05-02 06:24:48,060 - lava_callback - INFO - Starting async job submit for node 69f2f59be2fa3d7693ba95b5 2026-05-02 06:24:48,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:24:48,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f59be2fa3d7693ba95b5 HTTP/1.1" 200 None 2026-05-02 06:24:48,082 - lava_callback - DEBUG - Processing node 69f2f59be2fa3d7693ba95b5: baseline-arm64-clabbe 2026-05-02 06:24:48,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f59be2fa3d7693ba95b5 2026-05-02 06:24:48,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:24:48,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f59be2fa3d7693ba95b5 HTTP/1.1" 200 None 2026-05-02 06:24:48,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:24:48,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:24:48,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:24:48,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f59be2fa3d7693ba95b5 HTTP/1.1" 200 None 2026-05-02 06:24:48,138 - lava_callback - ERROR - Error processing callback for node 69f2f59be2fa3d7693ba95b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55024 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:25:28,840 - lava_callback - INFO - Received callback for node 69f2f5d6e2fa3d7693ba95d5 INFO: 172.20.0.1:55036 - "POST /node/69f2f5d6e2fa3d7693ba95d5 HTTP/1.0" 202 Accepted 2026-05-02 06:25:28,845 - lava_callback - INFO - Starting async job submit for node 69f2f5d6e2fa3d7693ba95d5 2026-05-02 06:25:28,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:25:28,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f5d6e2fa3d7693ba95d5 HTTP/1.1" 200 None 2026-05-02 06:25:28,867 - lava_callback - DEBUG - Processing node 69f2f5d6e2fa3d7693ba95d5: baseline-arm-baylibre 2026-05-02 06:25:28,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f5d6e2fa3d7693ba95d5 2026-05-02 06:25:28,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:25:28,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f5d6e2fa3d7693ba95d5 HTTP/1.1" 200 None 2026-05-02 06:25:28,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:25:28,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:25:28,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:25:28,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f5d6e2fa3d7693ba95d5 HTTP/1.1" 200 None 2026-05-02 06:25:28,912 - lava_callback - ERROR - Error processing callback for node 69f2f5d6e2fa3d7693ba95d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39192 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:26:28,869 - lava_callback - INFO - Received callback for node 69f2f600e2fa3d7693ba95e7 INFO: 172.20.0.1:39194 - "POST /node/69f2f600e2fa3d7693ba95e7 HTTP/1.0" 202 Accepted 2026-05-02 06:26:28,875 - lava_callback - INFO - Starting async job submit for node 69f2f600e2fa3d7693ba95e7 2026-05-02 06:26:28,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:26:28,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f600e2fa3d7693ba95e7 HTTP/1.1" 200 None 2026-05-02 06:26:28,890 - lava_callback - DEBUG - Processing node 69f2f600e2fa3d7693ba95e7: baseline-arm-baylibre 2026-05-02 06:26:28,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f600e2fa3d7693ba95e7 2026-05-02 06:26:28,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:26:28,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f600e2fa3d7693ba95e7 HTTP/1.1" 200 None 2026-05-02 06:26:28,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:26:28,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:26:28,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:26:28,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f600e2fa3d7693ba95e7 HTTP/1.1" 200 None 2026-05-02 06:26:28,938 - lava_callback - ERROR - Error processing callback for node 69f2f600e2fa3d7693ba95e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36460 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:27:08,192 - lava_callback - INFO - Received callback for node 69f2f637e2fa3d7693ba95fd INFO: 172.20.0.1:36476 - "POST /node/69f2f637e2fa3d7693ba95fd HTTP/1.0" 202 Accepted 2026-05-02 06:27:08,197 - lava_callback - INFO - Starting async job submit for node 69f2f637e2fa3d7693ba95fd 2026-05-02 06:27:08,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:27:08,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f637e2fa3d7693ba95fd HTTP/1.1" 200 None 2026-05-02 06:27:08,212 - lava_callback - DEBUG - Processing node 69f2f637e2fa3d7693ba95fd: baseline-arm64-clabbe 2026-05-02 06:27:08,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f637e2fa3d7693ba95fd 2026-05-02 06:27:08,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:27:08,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f637e2fa3d7693ba95fd HTTP/1.1" 200 None 2026-05-02 06:27:08,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:27:08,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:27:08,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:27:08,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f637e2fa3d7693ba95fd HTTP/1.1" 200 None 2026-05-02 06:27:08,281 - lava_callback - ERROR - Error processing callback for node 69f2f637e2fa3d7693ba95fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46644 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:28:09,077 - lava_callback - INFO - Received callback for node 69f2f673e2fa3d7693ba961c INFO: 172.20.0.1:46654 - "POST /node/69f2f673e2fa3d7693ba961c HTTP/1.0" 202 Accepted 2026-05-02 06:28:09,085 - lava_callback - INFO - Starting async job submit for node 69f2f673e2fa3d7693ba961c 2026-05-02 06:28:09,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:09,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f673e2fa3d7693ba961c HTTP/1.1" 200 None 2026-05-02 06:28:09,099 - lava_callback - DEBUG - Processing node 69f2f673e2fa3d7693ba961c: baseline-arm-baylibre 2026-05-02 06:28:09,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f673e2fa3d7693ba961c 2026-05-02 06:28:09,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:09,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f673e2fa3d7693ba961c HTTP/1.1" 200 None 2026-05-02 06:28:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:09,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:28:09,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:09,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f673e2fa3d7693ba961c HTTP/1.1" 200 None 2026-05-02 06:28:09,145 - lava_callback - ERROR - Error processing callback for node 69f2f673e2fa3d7693ba961c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49968 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:28:49,008 - lava_callback - INFO - Received callback for node 69f2f698e2fa3d7693ba9626 INFO: 172.20.0.1:49982 - "POST /node/69f2f698e2fa3d7693ba9626 HTTP/1.0" 202 Accepted 2026-05-02 06:28:49,013 - lava_callback - INFO - Starting async job submit for node 69f2f698e2fa3d7693ba9626 2026-05-02 06:28:49,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:49,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f698e2fa3d7693ba9626 HTTP/1.1" 200 None 2026-05-02 06:28:49,038 - lava_callback - DEBUG - Processing node 69f2f698e2fa3d7693ba9626: baseline-arm-baylibre 2026-05-02 06:28:49,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f698e2fa3d7693ba9626 2026-05-02 06:28:49,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:49,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f698e2fa3d7693ba9626 HTTP/1.1" 200 None 2026-05-02 06:28:49,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:49,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:28:49,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:28:49,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f698e2fa3d7693ba9626 HTTP/1.1" 200 None 2026-05-02 06:28:49,097 - lava_callback - ERROR - Error processing callback for node 69f2f698e2fa3d7693ba9626: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38238 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:29:48,384 - lava_callback - INFO - Received callback for node 69f2f6d2e2fa3d7693ba963a INFO: 172.20.0.1:38248 - "POST /node/69f2f6d2e2fa3d7693ba963a HTTP/1.0" 202 Accepted 2026-05-02 06:29:48,390 - lava_callback - INFO - Starting async job submit for node 69f2f6d2e2fa3d7693ba963a 2026-05-02 06:29:48,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:29:48,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f6d2e2fa3d7693ba963a HTTP/1.1" 200 None 2026-05-02 06:29:48,405 - lava_callback - DEBUG - Processing node 69f2f6d2e2fa3d7693ba963a: baseline-arm64-clabbe 2026-05-02 06:29:48,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f6d2e2fa3d7693ba963a 2026-05-02 06:29:48,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:29:48,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f6d2e2fa3d7693ba963a HTTP/1.1" 200 None 2026-05-02 06:29:48,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:29:48,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:29:48,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:29:48,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f6d2e2fa3d7693ba963a HTTP/1.1" 200 None 2026-05-02 06:29:48,469 - lava_callback - ERROR - Error processing callback for node 69f2f6d2e2fa3d7693ba963a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54436 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:30:49,152 - lava_callback - INFO - Received callback for node 69f2f70ce2fa3d7693ba9658 INFO: 172.20.0.1:54448 - "POST /node/69f2f70ce2fa3d7693ba9658 HTTP/1.0" 202 Accepted 2026-05-02 06:30:49,157 - lava_callback - INFO - Starting async job submit for node 69f2f70ce2fa3d7693ba9658 2026-05-02 06:30:49,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:30:49,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f70ce2fa3d7693ba9658 HTTP/1.1" 200 None 2026-05-02 06:30:49,173 - lava_callback - DEBUG - Processing node 69f2f70ce2fa3d7693ba9658: baseline-arm-baylibre 2026-05-02 06:30:49,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f70ce2fa3d7693ba9658 2026-05-02 06:30:49,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:30:49,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f70ce2fa3d7693ba9658 HTTP/1.1" 200 None 2026-05-02 06:30:49,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:30:49,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:30:49,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:30:49,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f70ce2fa3d7693ba9658 HTTP/1.1" 200 None 2026-05-02 06:30:49,219 - lava_callback - ERROR - Error processing callback for node 69f2f70ce2fa3d7693ba9658: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49866 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:31:29,149 - lava_callback - INFO - Received callback for node 69f2f734e2fa3d7693ba9666 INFO: 172.20.0.1:49872 - "POST /node/69f2f734e2fa3d7693ba9666 HTTP/1.0" 202 Accepted 2026-05-02 06:31:29,155 - lava_callback - INFO - Starting async job submit for node 69f2f734e2fa3d7693ba9666 2026-05-02 06:31:29,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:31:29,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f734e2fa3d7693ba9666 HTTP/1.1" 200 None 2026-05-02 06:31:29,178 - lava_callback - DEBUG - Processing node 69f2f734e2fa3d7693ba9666: baseline-arm-baylibre 2026-05-02 06:31:29,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f734e2fa3d7693ba9666 2026-05-02 06:31:29,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:31:29,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f734e2fa3d7693ba9666 HTTP/1.1" 200 None 2026-05-02 06:31:29,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:31:29,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:31:29,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:31:29,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f734e2fa3d7693ba9666 HTTP/1.1" 200 None 2026-05-02 06:31:29,223 - lava_callback - ERROR - Error processing callback for node 69f2f734e2fa3d7693ba9666: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60146 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:32:09,214 - lava_callback - INFO - Received callback for node 69f2f75be2fa3d7693ba9674 INFO: 172.20.0.1:60150 - "POST /node/69f2f75be2fa3d7693ba9674 HTTP/1.0" 202 Accepted 2026-05-02 06:32:09,219 - lava_callback - INFO - Starting async job submit for node 69f2f75be2fa3d7693ba9674 2026-05-02 06:32:09,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:32:09,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f75be2fa3d7693ba9674 HTTP/1.1" 200 None 2026-05-02 06:32:09,244 - lava_callback - DEBUG - Processing node 69f2f75be2fa3d7693ba9674: baseline-arm-baylibre 2026-05-02 06:32:09,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f75be2fa3d7693ba9674 2026-05-02 06:32:09,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:32:09,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f75be2fa3d7693ba9674 HTTP/1.1" 200 None 2026-05-02 06:32:09,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:32:09,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:32:09,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:32:09,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f75be2fa3d7693ba9674 HTTP/1.1" 200 None 2026-05-02 06:32:09,315 - lava_callback - ERROR - Error processing callback for node 69f2f75be2fa3d7693ba9674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60242 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:33:08,531 - lava_callback - INFO - Received callback for node 69f2f794e2fa3d7693ba9681 INFO: 172.20.0.1:60258 - "POST /node/69f2f794e2fa3d7693ba9681 HTTP/1.0" 202 Accepted 2026-05-02 06:33:08,537 - lava_callback - INFO - Starting async job submit for node 69f2f794e2fa3d7693ba9681 2026-05-02 06:33:08,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:33:08,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f794e2fa3d7693ba9681 HTTP/1.1" 200 None 2026-05-02 06:33:08,570 - lava_callback - DEBUG - Processing node 69f2f794e2fa3d7693ba9681: baseline-arm64-clabbe 2026-05-02 06:33:08,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f794e2fa3d7693ba9681 2026-05-02 06:33:08,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:33:08,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f794e2fa3d7693ba9681 HTTP/1.1" 200 None 2026-05-02 06:33:08,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:33:08,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:33:08,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:33:08,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f794e2fa3d7693ba9681 HTTP/1.1" 200 None 2026-05-02 06:33:08,628 - lava_callback - ERROR - Error processing callback for node 69f2f794e2fa3d7693ba9681: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:34:09,662 - lava_callback - INFO - Received callback for node 69f2f7cee2fa3d7693ba96a1 INFO: 172.20.0.1:46958 - "POST /node/69f2f7cee2fa3d7693ba96a1 HTTP/1.0" 202 Accepted 2026-05-02 06:34:09,668 - lava_callback - INFO - Starting async job submit for node 69f2f7cee2fa3d7693ba96a1 2026-05-02 06:34:09,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:09,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f7cee2fa3d7693ba96a1 HTTP/1.1" 200 None 2026-05-02 06:34:09,705 - lava_callback - DEBUG - Processing node 69f2f7cee2fa3d7693ba96a1: baseline-arm-baylibre 2026-05-02 06:34:09,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f7cee2fa3d7693ba96a1 2026-05-02 06:34:09,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:09,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f7cee2fa3d7693ba96a1 HTTP/1.1" 200 None 2026-05-02 06:34:09,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:09,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:34:09,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:09,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f7cee2fa3d7693ba96a1 HTTP/1.1" 200 None 2026-05-02 06:34:09,784 - lava_callback - ERROR - Error processing callback for node 69f2f7cee2fa3d7693ba96a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:34:49,361 - lava_callback - INFO - Received callback for node 69f2f7f5e2fa3d7693ba96b5 INFO: 172.20.0.1:60362 - "POST /node/69f2f7f5e2fa3d7693ba96b5 HTTP/1.0" 202 Accepted 2026-05-02 06:34:49,367 - lava_callback - INFO - Starting async job submit for node 69f2f7f5e2fa3d7693ba96b5 2026-05-02 06:34:49,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:49,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f7f5e2fa3d7693ba96b5 HTTP/1.1" 200 None 2026-05-02 06:34:49,382 - lava_callback - DEBUG - Processing node 69f2f7f5e2fa3d7693ba96b5: baseline-arm-baylibre 2026-05-02 06:34:49,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f7f5e2fa3d7693ba96b5 2026-05-02 06:34:49,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:49,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f7f5e2fa3d7693ba96b5 HTTP/1.1" 200 None 2026-05-02 06:34:49,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:49,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:34:49,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:34:49,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f7f5e2fa3d7693ba96b5 HTTP/1.1" 200 None 2026-05-02 06:34:49,436 - lava_callback - ERROR - Error processing callback for node 69f2f7f5e2fa3d7693ba96b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49808 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:35:28,643 - lava_callback - INFO - Received callback for node 69f2f82be2fa3d7693ba96c0 INFO: 172.20.0.1:49814 - "POST /node/69f2f82be2fa3d7693ba96c0 HTTP/1.0" 202 Accepted 2026-05-02 06:35:28,650 - lava_callback - INFO - Starting async job submit for node 69f2f82be2fa3d7693ba96c0 2026-05-02 06:35:28,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:35:28,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f82be2fa3d7693ba96c0 HTTP/1.1" 200 None 2026-05-02 06:35:28,665 - lava_callback - DEBUG - Processing node 69f2f82be2fa3d7693ba96c0: baseline-arm64-clabbe 2026-05-02 06:35:28,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f82be2fa3d7693ba96c0 2026-05-02 06:35:28,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:35:28,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f82be2fa3d7693ba96c0 HTTP/1.1" 200 None 2026-05-02 06:35:28,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:35:28,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:35:28,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:35:28,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f82be2fa3d7693ba96c0 HTTP/1.1" 200 None 2026-05-02 06:35:28,728 - lava_callback - ERROR - Error processing callback for node 69f2f82be2fa3d7693ba96c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:36:48,741 - lava_callback - INFO - Received callback for node 69f2f875e2fa3d7693ba96ee INFO: 172.20.0.1:55958 - "POST /node/69f2f875e2fa3d7693ba96ee HTTP/1.0" 202 Accepted 2026-05-02 06:36:48,746 - lava_callback - INFO - Starting async job submit for node 69f2f875e2fa3d7693ba96ee 2026-05-02 06:36:48,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:36:48,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f875e2fa3d7693ba96ee HTTP/1.1" 200 None 2026-05-02 06:36:48,761 - lava_callback - DEBUG - Processing node 69f2f875e2fa3d7693ba96ee: baseline-arm64-clabbe 2026-05-02 06:36:48,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f875e2fa3d7693ba96ee 2026-05-02 06:36:48,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:36:48,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f875e2fa3d7693ba96ee HTTP/1.1" 200 None 2026-05-02 06:36:48,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:36:48,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e668305534e972332b2 HTTP/1.1" 200 None 2026-05-02 06:36:48,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:36:48,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f875e2fa3d7693ba96ee HTTP/1.1" 200 None 2026-05-02 06:36:48,806 - lava_callback - ERROR - Error processing callback for node 69f2f875e2fa3d7693ba96ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33610 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:37:49,725 - lava_callback - INFO - Received callback for node 69f2f8ade2fa3d7693ba970e INFO: 172.20.0.1:43882 - "POST /node/69f2f8ade2fa3d7693ba970e HTTP/1.0" 202 Accepted 2026-05-02 06:37:49,730 - lava_callback - INFO - Starting async job submit for node 69f2f8ade2fa3d7693ba970e 2026-05-02 06:37:49,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:37:49,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f8ade2fa3d7693ba970e HTTP/1.1" 200 None 2026-05-02 06:37:49,745 - lava_callback - DEBUG - Processing node 69f2f8ade2fa3d7693ba970e: baseline-arm-baylibre 2026-05-02 06:37:49,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f8ade2fa3d7693ba970e 2026-05-02 06:37:49,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:37:49,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f8ade2fa3d7693ba970e HTTP/1.1" 200 None 2026-05-02 06:37:49,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:37:49,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:37:49,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:37:49,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f8ade2fa3d7693ba970e HTTP/1.1" 200 None 2026-05-02 06:37:49,792 - lava_callback - ERROR - Error processing callback for node 69f2f8ade2fa3d7693ba970e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37656 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:38:29,526 - lava_callback - INFO - Received callback for node 69f2f8d5e2fa3d7693ba971a INFO: 172.20.0.1:37666 - "POST /node/69f2f8d5e2fa3d7693ba971a HTTP/1.0" 202 Accepted 2026-05-02 06:38:29,531 - lava_callback - INFO - Starting async job submit for node 69f2f8d5e2fa3d7693ba971a 2026-05-02 06:38:29,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:38:29,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f8d5e2fa3d7693ba971a HTTP/1.1" 200 None 2026-05-02 06:38:29,546 - lava_callback - DEBUG - Processing node 69f2f8d5e2fa3d7693ba971a: baseline-arm-baylibre 2026-05-02 06:38:29,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f8d5e2fa3d7693ba971a 2026-05-02 06:38:29,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:38:29,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f8d5e2fa3d7693ba971a HTTP/1.1" 200 None 2026-05-02 06:38:29,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:38:29,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:38:29,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:38:29,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f8d5e2fa3d7693ba971a HTTP/1.1" 200 None 2026-05-02 06:38:29,596 - lava_callback - ERROR - Error processing callback for node 69f2f8d5e2fa3d7693ba971a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40736 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:39:09,673 - lava_callback - INFO - Received callback for node 69f2f8fce2fa3d7693ba9726 INFO: 172.20.0.1:40750 - "POST /node/69f2f8fce2fa3d7693ba9726 HTTP/1.0" 202 Accepted 2026-05-02 06:39:09,678 - lava_callback - INFO - Starting async job submit for node 69f2f8fce2fa3d7693ba9726 2026-05-02 06:39:09,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:09,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f8fce2fa3d7693ba9726 HTTP/1.1" 200 None 2026-05-02 06:39:09,694 - lava_callback - DEBUG - Processing node 69f2f8fce2fa3d7693ba9726: baseline-arm-baylibre 2026-05-02 06:39:09,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f8fce2fa3d7693ba9726 2026-05-02 06:39:09,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:09,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f8fce2fa3d7693ba9726 HTTP/1.1" 200 None 2026-05-02 06:39:09,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:09,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:39:09,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:09,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f8fce2fa3d7693ba9726 HTTP/1.1" 200 None 2026-05-02 06:39:09,770 - lava_callback - ERROR - Error processing callback for node 69f2f8fce2fa3d7693ba9726: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59912 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:39:49,637 - lava_callback - INFO - Received callback for node 69f2f924e2fa3d7693ba9733 INFO: 172.20.0.1:59926 - "POST /node/69f2f924e2fa3d7693ba9733 HTTP/1.0" 202 Accepted 2026-05-02 06:39:49,643 - lava_callback - INFO - Starting async job submit for node 69f2f924e2fa3d7693ba9733 2026-05-02 06:39:49,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:49,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f924e2fa3d7693ba9733 HTTP/1.1" 200 None 2026-05-02 06:39:49,658 - lava_callback - DEBUG - Processing node 69f2f924e2fa3d7693ba9733: baseline-arm-baylibre 2026-05-02 06:39:49,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f924e2fa3d7693ba9733 2026-05-02 06:39:49,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:49,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f924e2fa3d7693ba9733 HTTP/1.1" 200 None 2026-05-02 06:39:49,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:49,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:39:49,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:39:49,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f924e2fa3d7693ba9733 HTTP/1.1" 200 None 2026-05-02 06:39:49,705 - lava_callback - ERROR - Error processing callback for node 69f2f924e2fa3d7693ba9733: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60706 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:40:29,742 - lava_callback - INFO - Received callback for node 69f2f94de2fa3d7693ba973e INFO: 172.20.0.1:38670 - "POST /node/69f2f94de2fa3d7693ba973e HTTP/1.0" 202 Accepted 2026-05-02 06:40:29,748 - lava_callback - INFO - Starting async job submit for node 69f2f94de2fa3d7693ba973e 2026-05-02 06:40:29,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:40:29,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f94de2fa3d7693ba973e HTTP/1.1" 200 None 2026-05-02 06:40:29,780 - lava_callback - DEBUG - Processing node 69f2f94de2fa3d7693ba973e: baseline-arm-baylibre 2026-05-02 06:40:29,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f94de2fa3d7693ba973e 2026-05-02 06:40:29,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:40:29,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f94de2fa3d7693ba973e HTTP/1.1" 200 None 2026-05-02 06:40:29,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:40:29,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:40:29,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:40:29,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f94de2fa3d7693ba973e HTTP/1.1" 200 None 2026-05-02 06:40:29,831 - lava_callback - ERROR - Error processing callback for node 69f2f94de2fa3d7693ba973e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37620 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:41:09,731 - lava_callback - INFO - Received callback for node 69f2f974e2fa3d7693ba974a INFO: 172.20.0.1:58838 - "POST /node/69f2f974e2fa3d7693ba974a HTTP/1.0" 202 Accepted 2026-05-02 06:41:09,737 - lava_callback - INFO - Starting async job submit for node 69f2f974e2fa3d7693ba974a 2026-05-02 06:41:09,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:09,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f974e2fa3d7693ba974a HTTP/1.1" 200 None 2026-05-02 06:41:09,752 - lava_callback - DEBUG - Processing node 69f2f974e2fa3d7693ba974a: baseline-arm-baylibre 2026-05-02 06:41:09,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f974e2fa3d7693ba974a 2026-05-02 06:41:09,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:09,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f974e2fa3d7693ba974a HTTP/1.1" 200 None 2026-05-02 06:41:09,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:09,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:41:09,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:09,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f974e2fa3d7693ba974a HTTP/1.1" 200 None 2026-05-02 06:41:09,798 - lava_callback - ERROR - Error processing callback for node 69f2f974e2fa3d7693ba974a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35420 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:41:48,931 - lava_callback - INFO - Received callback for node 69f2f99de2fa3d7693ba975a INFO: 172.20.0.1:35432 - "POST /node/69f2f99de2fa3d7693ba975a HTTP/1.0" 202 Accepted 2026-05-02 06:41:48,936 - lava_callback - INFO - Starting async job submit for node 69f2f99de2fa3d7693ba975a 2026-05-02 06:41:48,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:48,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f99de2fa3d7693ba975a HTTP/1.1" 200 None 2026-05-02 06:41:48,951 - lava_callback - DEBUG - Processing node 69f2f99de2fa3d7693ba975a: baseline-arm-clabbe 2026-05-02 06:41:48,951 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f99de2fa3d7693ba975a 2026-05-02 06:41:48,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:48,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f99de2fa3d7693ba975a HTTP/1.1" 200 None 2026-05-02 06:41:48,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:48,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:41:48,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:41:48,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f99de2fa3d7693ba975a HTTP/1.1" 200 None 2026-05-02 06:41:48,995 - lava_callback - ERROR - Error processing callback for node 69f2f99de2fa3d7693ba975a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46624 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:42:28,965 - lava_callback - INFO - Received callback for node 69f2f9c4e2fa3d7693ba9765 INFO: 172.20.0.1:46628 - "POST /node/69f2f9c4e2fa3d7693ba9765 HTTP/1.0" 202 Accepted 2026-05-02 06:42:28,971 - lava_callback - INFO - Starting async job submit for node 69f2f9c4e2fa3d7693ba9765 2026-05-02 06:42:28,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:42:28,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f9c4e2fa3d7693ba9765 HTTP/1.1" 200 None 2026-05-02 06:42:28,994 - lava_callback - DEBUG - Processing node 69f2f9c4e2fa3d7693ba9765: baseline-arm-clabbe 2026-05-02 06:42:28,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f9c4e2fa3d7693ba9765 2026-05-02 06:42:28,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:42:29,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f9c4e2fa3d7693ba9765 HTTP/1.1" 200 None 2026-05-02 06:42:29,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:42:29,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:42:29,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:42:29,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f9c4e2fa3d7693ba9765 HTTP/1.1" 200 None 2026-05-02 06:42:29,046 - lava_callback - ERROR - Error processing callback for node 69f2f9c4e2fa3d7693ba9765: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44984 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:43:08,968 - lava_callback - INFO - Received callback for node 69f2f9ece2fa3d7693ba9771 INFO: 172.20.0.1:45000 - "POST /node/69f2f9ece2fa3d7693ba9771 HTTP/1.0" 202 Accepted 2026-05-02 06:43:08,973 - lava_callback - INFO - Starting async job submit for node 69f2f9ece2fa3d7693ba9771 2026-05-02 06:43:08,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:08,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f9ece2fa3d7693ba9771 HTTP/1.1" 200 None 2026-05-02 06:43:08,989 - lava_callback - DEBUG - Processing node 69f2f9ece2fa3d7693ba9771: baseline-arm-clabbe 2026-05-02 06:43:08,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2f9ece2fa3d7693ba9771 2026-05-02 06:43:08,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:09,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2f9ece2fa3d7693ba9771 HTTP/1.1" 200 None 2026-05-02 06:43:09,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:09,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:43:09,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:09,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2f9ece2fa3d7693ba9771 HTTP/1.1" 200 None 2026-05-02 06:43:09,047 - lava_callback - ERROR - Error processing callback for node 69f2f9ece2fa3d7693ba9771: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:43:49,018 - lava_callback - INFO - Received callback for node 69f2fa15e2fa3d7693ba9781 INFO: 172.20.0.1:40426 - "POST /node/69f2fa15e2fa3d7693ba9781 HTTP/1.0" 202 Accepted 2026-05-02 06:43:49,023 - lava_callback - INFO - Starting async job submit for node 69f2fa15e2fa3d7693ba9781 2026-05-02 06:43:49,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:49,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa15e2fa3d7693ba9781 HTTP/1.1" 200 None 2026-05-02 06:43:49,039 - lava_callback - DEBUG - Processing node 69f2fa15e2fa3d7693ba9781: baseline-arm-clabbe 2026-05-02 06:43:49,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fa15e2fa3d7693ba9781 2026-05-02 06:43:49,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:49,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa15e2fa3d7693ba9781 HTTP/1.1" 200 None 2026-05-02 06:43:49,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:49,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:43:49,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:43:49,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fa15e2fa3d7693ba9781 HTTP/1.1" 200 None 2026-05-02 06:43:49,085 - lava_callback - ERROR - Error processing callback for node 69f2fa15e2fa3d7693ba9781: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:44:29,057 - lava_callback - INFO - Received callback for node 69f2fa3de2fa3d7693ba978c INFO: 172.20.0.1:41398 - "POST /node/69f2fa3de2fa3d7693ba978c HTTP/1.0" 202 Accepted 2026-05-02 06:44:29,063 - lava_callback - INFO - Starting async job submit for node 69f2fa3de2fa3d7693ba978c 2026-05-02 06:44:29,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:44:29,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa3de2fa3d7693ba978c HTTP/1.1" 200 None 2026-05-02 06:44:29,078 - lava_callback - DEBUG - Processing node 69f2fa3de2fa3d7693ba978c: baseline-arm-clabbe 2026-05-02 06:44:29,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fa3de2fa3d7693ba978c 2026-05-02 06:44:29,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:44:29,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa3de2fa3d7693ba978c HTTP/1.1" 200 None 2026-05-02 06:44:29,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:44:29,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:44:29,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:44:29,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fa3de2fa3d7693ba978c HTTP/1.1" 200 None 2026-05-02 06:44:29,138 - lava_callback - ERROR - Error processing callback for node 69f2fa3de2fa3d7693ba978c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58416 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:45:09,058 - lava_callback - INFO - Received callback for node 69f2fa64e2fa3d7693ba9798 INFO: 172.20.0.1:58430 - "POST /node/69f2fa64e2fa3d7693ba9798 HTTP/1.0" 202 Accepted 2026-05-02 06:45:09,064 - lava_callback - INFO - Starting async job submit for node 69f2fa64e2fa3d7693ba9798 2026-05-02 06:45:09,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:09,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa64e2fa3d7693ba9798 HTTP/1.1" 200 None 2026-05-02 06:45:09,079 - lava_callback - DEBUG - Processing node 69f2fa64e2fa3d7693ba9798: baseline-arm-clabbe 2026-05-02 06:45:09,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fa64e2fa3d7693ba9798 2026-05-02 06:45:09,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:09,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa64e2fa3d7693ba9798 HTTP/1.1" 200 None 2026-05-02 06:45:09,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:09,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:45:09,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:09,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fa64e2fa3d7693ba9798 HTTP/1.1" 200 None 2026-05-02 06:45:09,123 - lava_callback - ERROR - Error processing callback for node 69f2fa64e2fa3d7693ba9798: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:45:49,140 - lava_callback - INFO - Received callback for node 69f2fa8be2fa3d7693ba97a3 INFO: 172.20.0.1:59712 - "POST /node/69f2fa8be2fa3d7693ba97a3 HTTP/1.0" 202 Accepted 2026-05-02 06:45:49,145 - lava_callback - INFO - Starting async job submit for node 69f2fa8be2fa3d7693ba97a3 2026-05-02 06:45:49,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:49,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa8be2fa3d7693ba97a3 HTTP/1.1" 200 None 2026-05-02 06:45:49,160 - lava_callback - DEBUG - Processing node 69f2fa8be2fa3d7693ba97a3: baseline-arm-clabbe 2026-05-02 06:45:49,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fa8be2fa3d7693ba97a3 2026-05-02 06:45:49,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:49,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fa8be2fa3d7693ba97a3 HTTP/1.1" 200 None 2026-05-02 06:45:49,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:49,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:45:49,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:45:49,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fa8be2fa3d7693ba97a3 HTTP/1.1" 200 None 2026-05-02 06:45:49,217 - lava_callback - ERROR - Error processing callback for node 69f2fa8be2fa3d7693ba97a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53282 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:46:29,146 - lava_callback - INFO - Received callback for node 69f2fab2e2fa3d7693ba97ae INFO: 172.20.0.1:53288 - "POST /node/69f2fab2e2fa3d7693ba97ae HTTP/1.0" 202 Accepted 2026-05-02 06:46:29,151 - lava_callback - INFO - Starting async job submit for node 69f2fab2e2fa3d7693ba97ae 2026-05-02 06:46:29,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:46:29,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fab2e2fa3d7693ba97ae HTTP/1.1" 200 None 2026-05-02 06:46:29,166 - lava_callback - DEBUG - Processing node 69f2fab2e2fa3d7693ba97ae: baseline-arm-clabbe 2026-05-02 06:46:29,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fab2e2fa3d7693ba97ae 2026-05-02 06:46:29,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:46:29,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fab2e2fa3d7693ba97ae HTTP/1.1" 200 None 2026-05-02 06:46:29,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:46:29,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:46:29,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:46:29,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fab2e2fa3d7693ba97ae HTTP/1.1" 200 None 2026-05-02 06:46:29,219 - lava_callback - ERROR - Error processing callback for node 69f2fab2e2fa3d7693ba97ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55220 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:47:09,197 - lava_callback - INFO - Received callback for node 69f2fadae2fa3d7693ba97b9 INFO: 172.20.0.1:55236 - "POST /node/69f2fadae2fa3d7693ba97b9 HTTP/1.0" 202 Accepted 2026-05-02 06:47:09,202 - lava_callback - INFO - Starting async job submit for node 69f2fadae2fa3d7693ba97b9 2026-05-02 06:47:09,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:09,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fadae2fa3d7693ba97b9 HTTP/1.1" 200 None 2026-05-02 06:47:09,217 - lava_callback - DEBUG - Processing node 69f2fadae2fa3d7693ba97b9: baseline-arm-clabbe 2026-05-02 06:47:09,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fadae2fa3d7693ba97b9 2026-05-02 06:47:09,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:09,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fadae2fa3d7693ba97b9 HTTP/1.1" 200 None 2026-05-02 06:47:09,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:09,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5e4d8305534e9723328e HTTP/1.1" 200 None 2026-05-02 06:47:09,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:09,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fadae2fa3d7693ba97b9 HTTP/1.1" 200 None 2026-05-02 06:47:09,271 - lava_callback - ERROR - Error processing callback for node 69f2fadae2fa3d7693ba97b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45010 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:47:30,362 - lava_callback - INFO - Received callback for node 69f2fafbe2fa3d7693ba97c4 INFO: 172.20.0.1:43890 - "POST /node/69f2fafbe2fa3d7693ba97c4 HTTP/1.0" 202 Accepted 2026-05-02 06:47:30,367 - lava_callback - INFO - Starting async job submit for node 69f2fafbe2fa3d7693ba97c4 2026-05-02 06:47:30,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:30,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fafbe2fa3d7693ba97c4 HTTP/1.1" 200 None 2026-05-02 06:47:30,382 - lava_callback - DEBUG - Processing node 69f2fafbe2fa3d7693ba97c4: baseline-arm-baylibre 2026-05-02 06:47:30,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fafbe2fa3d7693ba97c4 2026-05-02 06:47:30,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:30,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fafbe2fa3d7693ba97c4 HTTP/1.1" 200 None 2026-05-02 06:47:30,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:30,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:47:30,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:47:30,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fafbe2fa3d7693ba97c4 HTTP/1.1" 200 None 2026-05-02 06:47:30,428 - lava_callback - ERROR - Error processing callback for node 69f2fafbe2fa3d7693ba97c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40674 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:48:14,036 - lava_callback - INFO - Received callback for node 69f2fb21e2fa3d7693ba97cf INFO: 172.20.0.1:33736 - "POST /node/69f2fb21e2fa3d7693ba97cf HTTP/1.0" 202 Accepted 2026-05-02 06:48:14,042 - lava_callback - INFO - Starting async job submit for node 69f2fb21e2fa3d7693ba97cf 2026-05-02 06:48:14,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:14,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb21e2fa3d7693ba97cf HTTP/1.1" 200 None 2026-05-02 06:48:14,057 - lava_callback - DEBUG - Processing node 69f2fb21e2fa3d7693ba97cf: baseline-arm-baylibre 2026-05-02 06:48:14,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fb21e2fa3d7693ba97cf 2026-05-02 06:48:14,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:14,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb21e2fa3d7693ba97cf HTTP/1.1" 200 None 2026-05-02 06:48:14,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:14,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:48:14,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:14,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fb21e2fa3d7693ba97cf HTTP/1.1" 200 None 2026-05-02 06:48:14,105 - lava_callback - ERROR - Error processing callback for node 69f2fb21e2fa3d7693ba97cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33624 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:48:50,103 - lava_callback - INFO - Received callback for node 69f2fb48e2fa3d7693ba97dd INFO: 172.20.0.1:48130 - "POST /node/69f2fb48e2fa3d7693ba97dd HTTP/1.0" 202 Accepted 2026-05-02 06:48:50,108 - lava_callback - INFO - Starting async job submit for node 69f2fb48e2fa3d7693ba97dd 2026-05-02 06:48:50,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:50,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb48e2fa3d7693ba97dd HTTP/1.1" 200 None 2026-05-02 06:48:50,123 - lava_callback - DEBUG - Processing node 69f2fb48e2fa3d7693ba97dd: baseline-arm-baylibre 2026-05-02 06:48:50,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fb48e2fa3d7693ba97dd 2026-05-02 06:48:50,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:50,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb48e2fa3d7693ba97dd HTTP/1.1" 200 None 2026-05-02 06:48:50,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:50,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:48:50,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:48:50,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fb48e2fa3d7693ba97dd HTTP/1.1" 200 None 2026-05-02 06:48:50,182 - lava_callback - ERROR - Error processing callback for node 69f2fb48e2fa3d7693ba97dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41030 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:49:30,102 - lava_callback - INFO - Received callback for node 69f2fb6be2fa3d7693ba97e9 INFO: 172.20.0.1:37588 - "POST /node/69f2fb6be2fa3d7693ba97e9 HTTP/1.0" 202 Accepted 2026-05-02 06:49:30,107 - lava_callback - INFO - Starting async job submit for node 69f2fb6be2fa3d7693ba97e9 2026-05-02 06:49:30,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:49:30,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb6be2fa3d7693ba97e9 HTTP/1.1" 200 None 2026-05-02 06:49:30,122 - lava_callback - DEBUG - Processing node 69f2fb6be2fa3d7693ba97e9: baseline-arm-baylibre 2026-05-02 06:49:30,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fb6be2fa3d7693ba97e9 2026-05-02 06:49:30,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:49:30,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb6be2fa3d7693ba97e9 HTTP/1.1" 200 None 2026-05-02 06:49:30,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:49:30,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:49:30,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:49:30,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fb6be2fa3d7693ba97e9 HTTP/1.1" 200 None 2026-05-02 06:49:30,168 - lava_callback - ERROR - Error processing callback for node 69f2fb6be2fa3d7693ba97e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53180 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:50:10,268 - lava_callback - INFO - Received callback for node 69f2fb91e2fa3d7693ba97f6 INFO: 172.20.0.1:50596 - "POST /node/69f2fb91e2fa3d7693ba97f6 HTTP/1.0" 202 Accepted 2026-05-02 06:50:10,274 - lava_callback - INFO - Starting async job submit for node 69f2fb91e2fa3d7693ba97f6 2026-05-02 06:50:10,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:10,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb91e2fa3d7693ba97f6 HTTP/1.1" 200 None 2026-05-02 06:50:10,289 - lava_callback - DEBUG - Processing node 69f2fb91e2fa3d7693ba97f6: baseline-arm-baylibre 2026-05-02 06:50:10,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fb91e2fa3d7693ba97f6 2026-05-02 06:50:10,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:10,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fb91e2fa3d7693ba97f6 HTTP/1.1" 200 None 2026-05-02 06:50:10,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:10,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:50:10,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:10,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fb91e2fa3d7693ba97f6 HTTP/1.1" 200 None 2026-05-02 06:50:10,401 - lava_callback - ERROR - Error processing callback for node 69f2fb91e2fa3d7693ba97f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41880 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:50:50,164 - lava_callback - INFO - Received callback for node 69f2fbb5e2fa3d7693ba9807 INFO: 172.20.0.1:40592 - "POST /node/69f2fbb5e2fa3d7693ba9807 HTTP/1.0" 202 Accepted 2026-05-02 06:50:50,170 - lava_callback - INFO - Starting async job submit for node 69f2fbb5e2fa3d7693ba9807 2026-05-02 06:50:50,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:50,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fbb5e2fa3d7693ba9807 HTTP/1.1" 200 None 2026-05-02 06:50:50,200 - lava_callback - DEBUG - Processing node 69f2fbb5e2fa3d7693ba9807: baseline-arm-baylibre 2026-05-02 06:50:50,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fbb5e2fa3d7693ba9807 2026-05-02 06:50:50,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:50,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fbb5e2fa3d7693ba9807 HTTP/1.1" 200 None 2026-05-02 06:50:50,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:50,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:50:50,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:50:50,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fbb5e2fa3d7693ba9807 HTTP/1.1" 200 None 2026-05-02 06:50:50,247 - lava_callback - ERROR - Error processing callback for node 69f2fbb5e2fa3d7693ba9807: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:51:29,383 - lava_callback - INFO - Received callback for node 69f2fbdbe2fa3d7693ba981a INFO: 172.20.0.1:55128 - "POST /node/69f2fbdbe2fa3d7693ba981a HTTP/1.0" 202 Accepted 2026-05-02 06:51:29,389 - lava_callback - INFO - Starting async job submit for node 69f2fbdbe2fa3d7693ba981a 2026-05-02 06:51:29,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:29,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fbdbe2fa3d7693ba981a HTTP/1.1" 200 None 2026-05-02 06:51:29,409 - lava_callback - DEBUG - Processing node 69f2fbdbe2fa3d7693ba981a: baseline-arm-clabbe 2026-05-02 06:51:29,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fbdbe2fa3d7693ba981a 2026-05-02 06:51:29,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:29,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fbdbe2fa3d7693ba981a HTTP/1.1" 200 None 2026-05-02 06:51:29,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:29,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:51:29,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:29,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fbdbe2fa3d7693ba981a HTTP/1.1" 200 None 2026-05-02 06:51:29,455 - lava_callback - ERROR - Error processing callback for node 69f2fbdbe2fa3d7693ba981a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56432 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:51:49,382 - lava_callback - INFO - Received callback for node 69f2fc01e2fa3d7693ba9829 INFO: 172.20.0.1:56434 - "POST /node/69f2fc01e2fa3d7693ba9829 HTTP/1.0" 202 Accepted 2026-05-02 06:51:49,388 - lava_callback - INFO - Starting async job submit for node 69f2fc01e2fa3d7693ba9829 2026-05-02 06:51:49,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:49,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc01e2fa3d7693ba9829 HTTP/1.1" 200 None 2026-05-02 06:51:49,402 - lava_callback - DEBUG - Processing node 69f2fc01e2fa3d7693ba9829: baseline-arm-clabbe 2026-05-02 06:51:49,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fc01e2fa3d7693ba9829 2026-05-02 06:51:49,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:49,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc01e2fa3d7693ba9829 HTTP/1.1" 200 None 2026-05-02 06:51:49,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:49,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:51:49,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:51:49,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fc01e2fa3d7693ba9829 HTTP/1.1" 200 None 2026-05-02 06:51:49,447 - lava_callback - ERROR - Error processing callback for node 69f2fc01e2fa3d7693ba9829: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48200 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:52:29,412 - lava_callback - INFO - Received callback for node 69f2fc26e2fa3d7693ba983c INFO: 172.20.0.1:48204 - "POST /node/69f2fc26e2fa3d7693ba983c HTTP/1.0" 202 Accepted 2026-05-02 06:52:29,418 - lava_callback - INFO - Starting async job submit for node 69f2fc26e2fa3d7693ba983c 2026-05-02 06:52:29,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:52:29,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc26e2fa3d7693ba983c HTTP/1.1" 200 None 2026-05-02 06:52:29,433 - lava_callback - DEBUG - Processing node 69f2fc26e2fa3d7693ba983c: baseline-arm-clabbe 2026-05-02 06:52:29,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fc26e2fa3d7693ba983c 2026-05-02 06:52:29,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:52:29,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc26e2fa3d7693ba983c HTTP/1.1" 200 None 2026-05-02 06:52:29,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:52:29,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:52:29,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:52:29,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fc26e2fa3d7693ba983c HTTP/1.1" 200 None 2026-05-02 06:52:29,481 - lava_callback - ERROR - Error processing callback for node 69f2fc26e2fa3d7693ba983c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46132 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:53:09,407 - lava_callback - INFO - Received callback for node 69f2fc4ae2fa3d7693ba984c INFO: 172.20.0.1:46134 - "POST /node/69f2fc4ae2fa3d7693ba984c HTTP/1.0" 202 Accepted 2026-05-02 06:53:09,412 - lava_callback - INFO - Starting async job submit for node 69f2fc4ae2fa3d7693ba984c 2026-05-02 06:53:09,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:09,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc4ae2fa3d7693ba984c HTTP/1.1" 200 None 2026-05-02 06:53:09,427 - lava_callback - DEBUG - Processing node 69f2fc4ae2fa3d7693ba984c: baseline-arm-clabbe 2026-05-02 06:53:09,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fc4ae2fa3d7693ba984c 2026-05-02 06:53:09,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:09,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc4ae2fa3d7693ba984c HTTP/1.1" 200 None 2026-05-02 06:53:09,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:09,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:53:09,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:09,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fc4ae2fa3d7693ba984c HTTP/1.1" 200 None 2026-05-02 06:53:09,479 - lava_callback - ERROR - Error processing callback for node 69f2fc4ae2fa3d7693ba984c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50066 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:53:49,515 - lava_callback - INFO - Received callback for node 69f2fc71e2fa3d7693ba985e INFO: 172.20.0.1:50078 - "POST /node/69f2fc71e2fa3d7693ba985e HTTP/1.0" 202 Accepted 2026-05-02 06:53:49,520 - lava_callback - INFO - Starting async job submit for node 69f2fc71e2fa3d7693ba985e 2026-05-02 06:53:49,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:49,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc71e2fa3d7693ba985e HTTP/1.1" 200 None 2026-05-02 06:53:49,536 - lava_callback - DEBUG - Processing node 69f2fc71e2fa3d7693ba985e: baseline-arm-clabbe 2026-05-02 06:53:49,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fc71e2fa3d7693ba985e 2026-05-02 06:53:49,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:49,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc71e2fa3d7693ba985e HTTP/1.1" 200 None 2026-05-02 06:53:49,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:49,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:53:49,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:53:49,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fc71e2fa3d7693ba985e HTTP/1.1" 200 None 2026-05-02 06:53:49,587 - lava_callback - ERROR - Error processing callback for node 69f2fc71e2fa3d7693ba985e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60298 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:54:29,492 - lava_callback - INFO - Received callback for node 69f2fc9ae2fa3d7693ba986f INFO: 172.20.0.1:60300 - "POST /node/69f2fc9ae2fa3d7693ba986f HTTP/1.0" 202 Accepted 2026-05-02 06:54:29,498 - lava_callback - INFO - Starting async job submit for node 69f2fc9ae2fa3d7693ba986f 2026-05-02 06:54:29,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:54:29,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc9ae2fa3d7693ba986f HTTP/1.1" 200 None 2026-05-02 06:54:29,515 - lava_callback - DEBUG - Processing node 69f2fc9ae2fa3d7693ba986f: baseline-arm-clabbe 2026-05-02 06:54:29,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fc9ae2fa3d7693ba986f 2026-05-02 06:54:29,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:54:29,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fc9ae2fa3d7693ba986f HTTP/1.1" 200 None 2026-05-02 06:54:29,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:54:29,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:54:29,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:54:29,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fc9ae2fa3d7693ba986f HTTP/1.1" 200 None 2026-05-02 06:54:29,564 - lava_callback - ERROR - Error processing callback for node 69f2fc9ae2fa3d7693ba986f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59300 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:55:09,532 - lava_callback - INFO - Received callback for node 69f2fcc2e2fa3d7693ba9881 INFO: 172.20.0.1:59314 - "POST /node/69f2fcc2e2fa3d7693ba9881 HTTP/1.0" 202 Accepted 2026-05-02 06:55:09,537 - lava_callback - INFO - Starting async job submit for node 69f2fcc2e2fa3d7693ba9881 2026-05-02 06:55:09,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:09,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fcc2e2fa3d7693ba9881 HTTP/1.1" 200 None 2026-05-02 06:55:09,553 - lava_callback - DEBUG - Processing node 69f2fcc2e2fa3d7693ba9881: baseline-arm-clabbe 2026-05-02 06:55:09,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fcc2e2fa3d7693ba9881 2026-05-02 06:55:09,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:09,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fcc2e2fa3d7693ba9881 HTTP/1.1" 200 None 2026-05-02 06:55:09,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:09,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:55:09,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:09,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fcc2e2fa3d7693ba9881 HTTP/1.1" 200 None 2026-05-02 06:55:09,605 - lava_callback - ERROR - Error processing callback for node 69f2fcc2e2fa3d7693ba9881: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35716 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:55:49,540 - lava_callback - INFO - Received callback for node 69f2fce8e2fa3d7693ba9896 INFO: 172.20.0.1:35722 - "POST /node/69f2fce8e2fa3d7693ba9896 HTTP/1.0" 202 Accepted 2026-05-02 06:55:49,549 - lava_callback - INFO - Starting async job submit for node 69f2fce8e2fa3d7693ba9896 2026-05-02 06:55:49,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fce8e2fa3d7693ba9896 HTTP/1.1" 200 None 2026-05-02 06:55:49,565 - lava_callback - DEBUG - Processing node 69f2fce8e2fa3d7693ba9896: baseline-arm-clabbe 2026-05-02 06:55:49,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fce8e2fa3d7693ba9896 2026-05-02 06:55:49,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:49,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fce8e2fa3d7693ba9896 HTTP/1.1" 200 None 2026-05-02 06:55:49,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:49,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:55:49,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:55:49,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fce8e2fa3d7693ba9896 HTTP/1.1" 200 None 2026-05-02 06:55:49,611 - lava_callback - ERROR - Error processing callback for node 69f2fce8e2fa3d7693ba9896: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39704 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:56:29,582 - lava_callback - INFO - Received callback for node 69f2fd0fe2fa3d7693ba98a3 INFO: 172.20.0.1:39710 - "POST /node/69f2fd0fe2fa3d7693ba98a3 HTTP/1.0" 202 Accepted 2026-05-02 06:56:29,588 - lava_callback - INFO - Starting async job submit for node 69f2fd0fe2fa3d7693ba98a3 2026-05-02 06:56:29,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:56:29,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fd0fe2fa3d7693ba98a3 HTTP/1.1" 200 None 2026-05-02 06:56:29,604 - lava_callback - DEBUG - Processing node 69f2fd0fe2fa3d7693ba98a3: baseline-arm-clabbe 2026-05-02 06:56:29,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fd0fe2fa3d7693ba98a3 2026-05-02 06:56:29,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:56:29,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fd0fe2fa3d7693ba98a3 HTTP/1.1" 200 None 2026-05-02 06:56:29,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:56:29,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5df98305534e97233234 HTTP/1.1" 200 None 2026-05-02 06:56:29,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:56:29,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fd0fe2fa3d7693ba98a3 HTTP/1.1" 200 None 2026-05-02 06:56:29,649 - lava_callback - ERROR - Error processing callback for node 69f2fd0fe2fa3d7693ba98a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37274 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:58:10,627 - lava_callback - INFO - Received callback for node 69f2fd7fe2fa3d7693ba98ea INFO: 172.20.0.1:43040 - "POST /node/69f2fd7fe2fa3d7693ba98ea HTTP/1.0" 202 Accepted 2026-05-02 06:58:10,632 - lava_callback - INFO - Starting async job submit for node 69f2fd7fe2fa3d7693ba98ea 2026-05-02 06:58:10,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:58:10,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fd7fe2fa3d7693ba98ea HTTP/1.1" 200 None 2026-05-02 06:58:10,651 - lava_callback - DEBUG - Processing node 69f2fd7fe2fa3d7693ba98ea: baseline-arm64-baylibre 2026-05-02 06:58:10,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fd7fe2fa3d7693ba98ea 2026-05-02 06:58:10,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:58:10,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fd7fe2fa3d7693ba98ea HTTP/1.1" 200 None 2026-05-02 06:58:10,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:58:10,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:58:10,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:58:10,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fd7fe2fa3d7693ba98ea HTTP/1.1" 200 None 2026-05-02 06:58:10,698 - lava_callback - ERROR - Error processing callback for node 69f2fd7fe2fa3d7693ba98ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56286 - "GET / HTTP/1.0" 200 OK 2026-05-02 06:59:50,740 - lava_callback - INFO - Received callback for node 69f2fdd3e2fa3d7693ba9913 INFO: 172.20.0.1:36220 - "POST /node/69f2fdd3e2fa3d7693ba9913 HTTP/1.0" 202 Accepted 2026-05-02 06:59:50,745 - lava_callback - INFO - Starting async job submit for node 69f2fdd3e2fa3d7693ba9913 2026-05-02 06:59:50,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:59:50,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fdd3e2fa3d7693ba9913 HTTP/1.1" 200 None 2026-05-02 06:59:50,760 - lava_callback - DEBUG - Processing node 69f2fdd3e2fa3d7693ba9913: baseline-arm64-baylibre 2026-05-02 06:59:50,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fdd3e2fa3d7693ba9913 2026-05-02 06:59:50,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:59:50,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fdd3e2fa3d7693ba9913 HTTP/1.1" 200 None 2026-05-02 06:59:50,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:59:50,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 06:59:50,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 06:59:50,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fdd3e2fa3d7693ba9913 HTTP/1.1" 200 None 2026-05-02 06:59:50,814 - lava_callback - ERROR - Error processing callback for node 69f2fdd3e2fa3d7693ba9913: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52876 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:01:10,815 - lava_callback - INFO - Received callback for node 69f2fe2ee2fa3d7693ba9931 INFO: 172.20.0.1:55516 - "POST /node/69f2fe2ee2fa3d7693ba9931 HTTP/1.0" 202 Accepted 2026-05-02 07:01:10,821 - lava_callback - INFO - Starting async job submit for node 69f2fe2ee2fa3d7693ba9931 2026-05-02 07:01:10,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:01:10,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fe2ee2fa3d7693ba9931 HTTP/1.1" 200 None 2026-05-02 07:01:10,838 - lava_callback - DEBUG - Processing node 69f2fe2ee2fa3d7693ba9931: baseline-arm64-baylibre 2026-05-02 07:01:10,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fe2ee2fa3d7693ba9931 2026-05-02 07:01:10,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:01:10,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fe2ee2fa3d7693ba9931 HTTP/1.1" 200 None 2026-05-02 07:01:10,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:01:10,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:01:10,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:01:10,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fe2ee2fa3d7693ba9931 HTTP/1.1" 200 None 2026-05-02 07:01:10,884 - lava_callback - ERROR - Error processing callback for node 69f2fe2ee2fa3d7693ba9931: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:02:31,145 - lava_callback - INFO - Received callback for node 69f2fe83e2fa3d7693ba9953 INFO: 172.20.0.1:34148 - "POST /node/69f2fe83e2fa3d7693ba9953 HTTP/1.0" 202 Accepted 2026-05-02 07:02:31,151 - lava_callback - INFO - Starting async job submit for node 69f2fe83e2fa3d7693ba9953 2026-05-02 07:02:31,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:02:31,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fe83e2fa3d7693ba9953 HTTP/1.1" 200 None 2026-05-02 07:02:31,166 - lava_callback - DEBUG - Processing node 69f2fe83e2fa3d7693ba9953: baseline-arm64-baylibre 2026-05-02 07:02:31,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fe83e2fa3d7693ba9953 2026-05-02 07:02:31,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:02:31,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fe83e2fa3d7693ba9953 HTTP/1.1" 200 None 2026-05-02 07:02:31,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:02:31,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:02:31,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:02:31,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fe83e2fa3d7693ba9953 HTTP/1.1" 200 None 2026-05-02 07:02:31,214 - lava_callback - ERROR - Error processing callback for node 69f2fe83e2fa3d7693ba9953: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50252 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:04:10,973 - lava_callback - INFO - Received callback for node 69f2fed8e2fa3d7693ba9974 INFO: 172.20.0.1:55816 - "POST /node/69f2fed8e2fa3d7693ba9974 HTTP/1.0" 202 Accepted 2026-05-02 07:04:10,978 - lava_callback - INFO - Starting async job submit for node 69f2fed8e2fa3d7693ba9974 2026-05-02 07:04:10,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:04:10,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fed8e2fa3d7693ba9974 HTTP/1.1" 200 None 2026-05-02 07:04:10,993 - lava_callback - DEBUG - Processing node 69f2fed8e2fa3d7693ba9974: baseline-arm64-baylibre 2026-05-02 07:04:10,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2fed8e2fa3d7693ba9974 2026-05-02 07:04:10,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:04:11,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2fed8e2fa3d7693ba9974 HTTP/1.1" 200 None 2026-05-02 07:04:11,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:04:11,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:04:11,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:04:11,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2fed8e2fa3d7693ba9974 HTTP/1.1" 200 None 2026-05-02 07:04:11,038 - lava_callback - ERROR - Error processing callback for node 69f2fed8e2fa3d7693ba9974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:05:31,060 - lava_callback - INFO - Received callback for node 69f2ff2de2fa3d7693ba9998 INFO: 172.20.0.1:58432 - "POST /node/69f2ff2de2fa3d7693ba9998 HTTP/1.0" 202 Accepted 2026-05-02 07:05:31,065 - lava_callback - INFO - Starting async job submit for node 69f2ff2de2fa3d7693ba9998 2026-05-02 07:05:31,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:05:31,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ff2de2fa3d7693ba9998 HTTP/1.1" 200 None 2026-05-02 07:05:31,081 - lava_callback - DEBUG - Processing node 69f2ff2de2fa3d7693ba9998: baseline-arm64-baylibre 2026-05-02 07:05:31,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ff2de2fa3d7693ba9998 2026-05-02 07:05:31,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:05:31,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ff2de2fa3d7693ba9998 HTTP/1.1" 200 None 2026-05-02 07:05:31,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:05:31,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:05:31,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:05:31,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ff2de2fa3d7693ba9998 HTTP/1.1" 200 None 2026-05-02 07:05:31,133 - lava_callback - ERROR - Error processing callback for node 69f2ff2de2fa3d7693ba9998: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48538 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:07:51,156 - lava_callback - INFO - Received callback for node 69f2ffbae2fa3d7693ba99d6 INFO: 172.20.0.1:48478 - "POST /node/69f2ffbae2fa3d7693ba99d6 HTTP/1.0" 202 Accepted 2026-05-02 07:07:51,161 - lava_callback - INFO - Starting async job submit for node 69f2ffbae2fa3d7693ba99d6 2026-05-02 07:07:51,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:07:51,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ffbae2fa3d7693ba99d6 HTTP/1.1" 200 None 2026-05-02 07:07:51,177 - lava_callback - DEBUG - Processing node 69f2ffbae2fa3d7693ba99d6: baseline-arm-baylibre 2026-05-02 07:07:51,177 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f2ffbae2fa3d7693ba99d6 2026-05-02 07:07:51,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:07:51,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f2ffbae2fa3d7693ba99d6 HTTP/1.1" 200 None 2026-05-02 07:07:51,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:07:51,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf5199260e707730df4a HTTP/1.1" 200 None 2026-05-02 07:07:51,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:07:51,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f2ffbae2fa3d7693ba99d6 HTTP/1.1" 200 None 2026-05-02 07:07:51,228 - lava_callback - ERROR - Error processing callback for node 69f2ffbae2fa3d7693ba99d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46140 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:12:51,420 - lava_callback - INFO - Received callback for node 69f300e0e2fa3d7693ba9a85 INFO: 172.20.0.1:45476 - "POST /node/69f300e0e2fa3d7693ba9a85 HTTP/1.0" 202 Accepted 2026-05-02 07:12:51,426 - lava_callback - INFO - Starting async job submit for node 69f300e0e2fa3d7693ba9a85 2026-05-02 07:12:51,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:12:51,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f300e0e2fa3d7693ba9a85 HTTP/1.1" 200 None 2026-05-02 07:12:51,442 - lava_callback - DEBUG - Processing node 69f300e0e2fa3d7693ba9a85: baseline-arm64-baylibre 2026-05-02 07:12:51,442 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f300e0e2fa3d7693ba9a85 2026-05-02 07:12:51,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:12:51,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f300e0e2fa3d7693ba9a85 HTTP/1.1" 200 None 2026-05-02 07:12:51,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:12:51,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:12:51,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:12:51,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f300e0e2fa3d7693ba9a85 HTTP/1.1" 200 None 2026-05-02 07:12:51,495 - lava_callback - ERROR - Error processing callback for node 69f300e0e2fa3d7693ba9a85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40836 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:14:11,514 - lava_callback - INFO - Received callback for node 69f30135e2fa3d7693ba9aa3 INFO: 172.20.0.1:50838 - "POST /node/69f30135e2fa3d7693ba9aa3 HTTP/1.0" 202 Accepted 2026-05-02 07:14:11,519 - lava_callback - INFO - Starting async job submit for node 69f30135e2fa3d7693ba9aa3 2026-05-02 07:14:11,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:14:11,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30135e2fa3d7693ba9aa3 HTTP/1.1" 200 None 2026-05-02 07:14:11,535 - lava_callback - DEBUG - Processing node 69f30135e2fa3d7693ba9aa3: baseline-arm64-baylibre 2026-05-02 07:14:11,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30135e2fa3d7693ba9aa3 2026-05-02 07:14:11,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:14:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30135e2fa3d7693ba9aa3 HTTP/1.1" 200 None 2026-05-02 07:14:11,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:14:11,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:14:11,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:14:11,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30135e2fa3d7693ba9aa3 HTTP/1.1" 200 None 2026-05-02 07:14:11,583 - lava_callback - ERROR - Error processing callback for node 69f30135e2fa3d7693ba9aa3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59514 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:15:31,588 - lava_callback - INFO - Received callback for node 69f3018be2fa3d7693ba9ac4 INFO: 172.20.0.1:53608 - "POST /node/69f3018be2fa3d7693ba9ac4 HTTP/1.0" 202 Accepted 2026-05-02 07:15:31,594 - lava_callback - INFO - Starting async job submit for node 69f3018be2fa3d7693ba9ac4 2026-05-02 07:15:31,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:15:31,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3018be2fa3d7693ba9ac4 HTTP/1.1" 200 None 2026-05-02 07:15:31,625 - lava_callback - DEBUG - Processing node 69f3018be2fa3d7693ba9ac4: baseline-arm64-baylibre 2026-05-02 07:15:31,625 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f3018be2fa3d7693ba9ac4 2026-05-02 07:15:31,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:15:31,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3018be2fa3d7693ba9ac4 HTTP/1.1" 200 None 2026-05-02 07:15:31,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:15:31,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:15:31,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:15:31,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f3018be2fa3d7693ba9ac4 HTTP/1.1" 200 None 2026-05-02 07:15:31,671 - lava_callback - ERROR - Error processing callback for node 69f3018be2fa3d7693ba9ac4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60996 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-02 07:17:15,619 - lava_callback - INFO - Received callback for node 69f301e7e2fa3d7693ba9ae2 INFO: 172.20.0.1:60998 - "POST /node/69f301e7e2fa3d7693ba9ae2 HTTP/1.0" 202 Accepted 2026-05-02 07:17:15,625 - lava_callback - INFO - Starting async job submit for node 69f301e7e2fa3d7693ba9ae2 2026-05-02 07:17:15,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:17:15,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f301e7e2fa3d7693ba9ae2 HTTP/1.1" 200 None 2026-05-02 07:17:15,641 - lava_callback - DEBUG - Processing node 69f301e7e2fa3d7693ba9ae2: baseline-arm64-baylibre 2026-05-02 07:17:15,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f301e7e2fa3d7693ba9ae2 2026-05-02 07:17:15,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:17:15,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f301e7e2fa3d7693ba9ae2 HTTP/1.1" 200 None 2026-05-02 07:17:15,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:17:15,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:17:15,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:17:15,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f301e7e2fa3d7693ba9ae2 HTTP/1.1" 200 None 2026-05-02 07:17:15,693 - lava_callback - ERROR - Error processing callback for node 69f301e7e2fa3d7693ba9ae2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48474 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:18:31,792 - lava_callback - INFO - Received callback for node 69f30241e2fa3d7693ba9b03 INFO: 172.20.0.1:57954 - "POST /node/69f30241e2fa3d7693ba9b03 HTTP/1.0" 202 Accepted 2026-05-02 07:18:31,798 - lava_callback - INFO - Starting async job submit for node 69f30241e2fa3d7693ba9b03 2026-05-02 07:18:31,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:18:31,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30241e2fa3d7693ba9b03 HTTP/1.1" 200 None 2026-05-02 07:18:31,817 - lava_callback - DEBUG - Processing node 69f30241e2fa3d7693ba9b03: baseline-arm64-baylibre 2026-05-02 07:18:31,818 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30241e2fa3d7693ba9b03 2026-05-02 07:18:31,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:18:31,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30241e2fa3d7693ba9b03 HTTP/1.1" 200 None 2026-05-02 07:18:31,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:18:31,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:18:31,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:18:31,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30241e2fa3d7693ba9b03 HTTP/1.1" 200 None 2026-05-02 07:18:31,873 - lava_callback - ERROR - Error processing callback for node 69f30241e2fa3d7693ba9b03: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:20:11,853 - lava_callback - INFO - Received callback for node 69f30297e2fa3d7693ba9b24 INFO: 172.20.0.1:53218 - "POST /node/69f30297e2fa3d7693ba9b24 HTTP/1.0" 202 Accepted 2026-05-02 07:20:11,863 - lava_callback - INFO - Starting async job submit for node 69f30297e2fa3d7693ba9b24 2026-05-02 07:20:11,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:20:11,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30297e2fa3d7693ba9b24 HTTP/1.1" 200 None 2026-05-02 07:20:11,881 - lava_callback - DEBUG - Processing node 69f30297e2fa3d7693ba9b24: baseline-arm64-baylibre 2026-05-02 07:20:11,882 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30297e2fa3d7693ba9b24 2026-05-02 07:20:11,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:20:11,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30297e2fa3d7693ba9b24 HTTP/1.1" 200 None 2026-05-02 07:20:11,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:20:11,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:20:11,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:20:11,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30297e2fa3d7693ba9b24 HTTP/1.1" 200 None 2026-05-02 07:20:11,952 - lava_callback - ERROR - Error processing callback for node 69f30297e2fa3d7693ba9b24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51036 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:21:50,904 - lava_callback - INFO - Received callback for node 69f302fde2fa3d7693ba9b53 INFO: 172.20.0.1:56916 - "POST /node/69f302fde2fa3d7693ba9b53 HTTP/1.0" 202 Accepted 2026-05-02 07:21:50,910 - lava_callback - INFO - Starting async job submit for node 69f302fde2fa3d7693ba9b53 2026-05-02 07:21:50,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:21:50,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f302fde2fa3d7693ba9b53 HTTP/1.1" 200 None 2026-05-02 07:21:50,925 - lava_callback - DEBUG - Processing node 69f302fde2fa3d7693ba9b53: baseline-arm64-clabbe 2026-05-02 07:21:50,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f302fde2fa3d7693ba9b53 2026-05-02 07:21:50,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:21:50,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f302fde2fa3d7693ba9b53 HTTP/1.1" 200 None 2026-05-02 07:21:50,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:21:50,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:21:50,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:21:50,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f302fde2fa3d7693ba9b53 HTTP/1.1" 200 None 2026-05-02 07:21:50,977 - lava_callback - ERROR - Error processing callback for node 69f302fde2fa3d7693ba9b53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59822 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:23:10,800 - lava_callback - INFO - Received callback for node 69f30353e2fa3d7693ba9b68 INFO: 172.20.0.1:46580 - "POST /node/69f30353e2fa3d7693ba9b68 HTTP/1.0" 202 Accepted 2026-05-02 07:23:10,806 - lava_callback - INFO - Starting async job submit for node 69f30353e2fa3d7693ba9b68 2026-05-02 07:23:10,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:23:10,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30353e2fa3d7693ba9b68 HTTP/1.1" 200 None 2026-05-02 07:23:10,823 - lava_callback - DEBUG - Processing node 69f30353e2fa3d7693ba9b68: baseline-arm64-clabbe 2026-05-02 07:23:10,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30353e2fa3d7693ba9b68 2026-05-02 07:23:10,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:23:10,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30353e2fa3d7693ba9b68 HTTP/1.1" 200 None 2026-05-02 07:23:10,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:23:10,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:23:10,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:23:10,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30353e2fa3d7693ba9b68 HTTP/1.1" 200 None 2026-05-02 07:23:10,867 - lava_callback - ERROR - Error processing callback for node 69f30353e2fa3d7693ba9b68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37852 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:24:30,994 - lava_callback - INFO - Received callback for node 69f303aae2fa3d7693ba9b8c INFO: 172.20.0.1:55012 - "POST /node/69f303aae2fa3d7693ba9b8c HTTP/1.0" 202 Accepted 2026-05-02 07:24:30,999 - lava_callback - INFO - Starting async job submit for node 69f303aae2fa3d7693ba9b8c 2026-05-02 07:24:31,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:24:31,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f303aae2fa3d7693ba9b8c HTTP/1.1" 200 None 2026-05-02 07:24:31,015 - lava_callback - DEBUG - Processing node 69f303aae2fa3d7693ba9b8c: baseline-arm64-clabbe 2026-05-02 07:24:31,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f303aae2fa3d7693ba9b8c 2026-05-02 07:24:31,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:24:31,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f303aae2fa3d7693ba9b8c HTTP/1.1" 200 None 2026-05-02 07:24:31,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:24:31,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:24:31,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:24:31,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f303aae2fa3d7693ba9b8c HTTP/1.1" 200 None 2026-05-02 07:24:31,073 - lava_callback - ERROR - Error processing callback for node 69f303aae2fa3d7693ba9b8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42290 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:26:10,959 - lava_callback - INFO - Received callback for node 69f303ffe2fa3d7693ba9bad INFO: 172.20.0.1:48184 - "POST /node/69f303ffe2fa3d7693ba9bad HTTP/1.0" 202 Accepted 2026-05-02 07:26:10,964 - lava_callback - INFO - Starting async job submit for node 69f303ffe2fa3d7693ba9bad 2026-05-02 07:26:10,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:26:10,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f303ffe2fa3d7693ba9bad HTTP/1.1" 200 None 2026-05-02 07:26:10,979 - lava_callback - DEBUG - Processing node 69f303ffe2fa3d7693ba9bad: baseline-arm64-clabbe 2026-05-02 07:26:10,979 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f303ffe2fa3d7693ba9bad 2026-05-02 07:26:10,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:26:10,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f303ffe2fa3d7693ba9bad HTTP/1.1" 200 None 2026-05-02 07:26:10,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:26:11,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:26:11,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:26:11,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f303ffe2fa3d7693ba9bad HTTP/1.1" 200 None 2026-05-02 07:26:11,031 - lava_callback - ERROR - Error processing callback for node 69f303ffe2fa3d7693ba9bad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54440 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:27:30,991 - lava_callback - INFO - Received callback for node 69f30455e2fa3d7693ba9bcd INFO: 172.20.0.1:42366 - "POST /node/69f30455e2fa3d7693ba9bcd HTTP/1.0" 202 Accepted 2026-05-02 07:27:30,997 - lava_callback - INFO - Starting async job submit for node 69f30455e2fa3d7693ba9bcd 2026-05-02 07:27:30,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:27:31,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30455e2fa3d7693ba9bcd HTTP/1.1" 200 None 2026-05-02 07:27:31,013 - lava_callback - DEBUG - Processing node 69f30455e2fa3d7693ba9bcd: baseline-arm64-clabbe 2026-05-02 07:27:31,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30455e2fa3d7693ba9bcd 2026-05-02 07:27:31,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:27:31,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30455e2fa3d7693ba9bcd HTTP/1.1" 200 None 2026-05-02 07:27:31,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:27:31,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:27:31,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:27:31,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30455e2fa3d7693ba9bcd HTTP/1.1" 200 None 2026-05-02 07:27:31,068 - lava_callback - ERROR - Error processing callback for node 69f30455e2fa3d7693ba9bcd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:28:51,057 - lava_callback - INFO - Received callback for node 69f304aae2fa3d7693ba9bee INFO: 172.20.0.1:49758 - "POST /node/69f304aae2fa3d7693ba9bee HTTP/1.0" 202 Accepted 2026-05-02 07:28:51,062 - lava_callback - INFO - Starting async job submit for node 69f304aae2fa3d7693ba9bee 2026-05-02 07:28:51,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:28:51,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f304aae2fa3d7693ba9bee HTTP/1.1" 200 None 2026-05-02 07:28:51,077 - lava_callback - DEBUG - Processing node 69f304aae2fa3d7693ba9bee: baseline-arm64-clabbe 2026-05-02 07:28:51,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f304aae2fa3d7693ba9bee 2026-05-02 07:28:51,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:28:51,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f304aae2fa3d7693ba9bee HTTP/1.1" 200 None 2026-05-02 07:28:51,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:28:51,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:28:51,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:28:51,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f304aae2fa3d7693ba9bee HTTP/1.1" 200 None 2026-05-02 07:28:51,138 - lava_callback - ERROR - Error processing callback for node 69f304aae2fa3d7693ba9bee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59176 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:30:11,235 - lava_callback - INFO - Received callback for node 69f304fce2fa3d7693ba9c1e INFO: 172.20.0.1:40114 - "POST /node/69f304fce2fa3d7693ba9c1e HTTP/1.0" 202 Accepted 2026-05-02 07:30:11,245 - lava_callback - INFO - Starting async job submit for node 69f304fce2fa3d7693ba9c1e 2026-05-02 07:30:11,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:30:11,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f304fce2fa3d7693ba9c1e HTTP/1.1" 200 None 2026-05-02 07:30:11,262 - lava_callback - DEBUG - Processing node 69f304fce2fa3d7693ba9c1e: baseline-arm64-clabbe 2026-05-02 07:30:11,262 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f304fce2fa3d7693ba9c1e 2026-05-02 07:30:11,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:30:11,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f304fce2fa3d7693ba9c1e HTTP/1.1" 200 None 2026-05-02 07:30:11,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:30:11,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ef5eb98305534e97233321 HTTP/1.1" 200 None 2026-05-02 07:30:11,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:30:11,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f304fce2fa3d7693ba9c1e HTTP/1.1" 200 None 2026-05-02 07:30:11,333 - lava_callback - ERROR - Error processing callback for node 69f304fce2fa3d7693ba9c1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57198 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:37:52,923 - lava_callback - INFO - Received callback for node 69f306cee2fa3d7693ba9ccc INFO: 172.20.0.1:46856 - "POST /node/69f306cee2fa3d7693ba9ccc HTTP/1.0" 202 Accepted 2026-05-02 07:37:52,929 - lava_callback - INFO - Starting async job submit for node 69f306cee2fa3d7693ba9ccc 2026-05-02 07:37:52,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:37:52,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f306cee2fa3d7693ba9ccc HTTP/1.1" 200 None 2026-05-02 07:37:52,943 - lava_callback - DEBUG - Processing node 69f306cee2fa3d7693ba9ccc: baseline-arm64-baylibre 2026-05-02 07:37:52,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f306cee2fa3d7693ba9ccc 2026-05-02 07:37:52,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:37:52,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f306cee2fa3d7693ba9ccc HTTP/1.1" 200 None 2026-05-02 07:37:52,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:37:52,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f45f5ce4820821ef7fa HTTP/1.1" 200 None 2026-05-02 07:37:52,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:37:52,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f306cee2fa3d7693ba9ccc HTTP/1.1" 200 None 2026-05-02 07:37:52,990 - lava_callback - ERROR - Error processing callback for node 69f306cee2fa3d7693ba9ccc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33486 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:39:36,885 - lava_callback - INFO - Received callback for node 69f30724e2fa3d7693ba9ced INFO: 172.20.0.1:33500 - "POST /node/69f30724e2fa3d7693ba9ced HTTP/1.0" 202 Accepted 2026-05-02 07:39:36,891 - lava_callback - INFO - Starting async job submit for node 69f30724e2fa3d7693ba9ced 2026-05-02 07:39:36,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:39:36,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30724e2fa3d7693ba9ced HTTP/1.1" 200 None 2026-05-02 07:39:36,906 - lava_callback - DEBUG - Processing node 69f30724e2fa3d7693ba9ced: baseline-arm64-baylibre 2026-05-02 07:39:36,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30724e2fa3d7693ba9ced 2026-05-02 07:39:36,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:39:36,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30724e2fa3d7693ba9ced HTTP/1.1" 200 None 2026-05-02 07:39:36,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:39:36,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f45f5ce4820821ef7fa HTTP/1.1" 200 None 2026-05-02 07:39:36,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:39:36,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30724e2fa3d7693ba9ced HTTP/1.1" 200 None 2026-05-02 07:39:36,954 - lava_callback - ERROR - Error processing callback for node 69f30724e2fa3d7693ba9ced: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39882 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:40:53,048 - lava_callback - INFO - Received callback for node 69f30778e2fa3d7693ba9d10 INFO: 172.20.0.1:47984 - "POST /node/69f30778e2fa3d7693ba9d10 HTTP/1.0" 202 Accepted 2026-05-02 07:40:53,054 - lava_callback - INFO - Starting async job submit for node 69f30778e2fa3d7693ba9d10 2026-05-02 07:40:53,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:40:53,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30778e2fa3d7693ba9d10 HTTP/1.1" 200 None 2026-05-02 07:40:53,069 - lava_callback - DEBUG - Processing node 69f30778e2fa3d7693ba9d10: baseline-arm64-baylibre 2026-05-02 07:40:53,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30778e2fa3d7693ba9d10 2026-05-02 07:40:53,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:40:53,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30778e2fa3d7693ba9d10 HTTP/1.1" 200 None 2026-05-02 07:40:53,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:40:53,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f45f5ce4820821ef7fa HTTP/1.1" 200 None 2026-05-02 07:40:53,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:40:53,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30778e2fa3d7693ba9d10 HTTP/1.1" 200 None 2026-05-02 07:40:53,114 - lava_callback - ERROR - Error processing callback for node 69f30778e2fa3d7693ba9d10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42342 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:42:13,127 - lava_callback - INFO - Received callback for node 69f307cbe2fa3d7693ba9d2e INFO: 172.20.0.1:53288 - "POST /node/69f307cbe2fa3d7693ba9d2e HTTP/1.0" 202 Accepted 2026-05-02 07:42:13,133 - lava_callback - INFO - Starting async job submit for node 69f307cbe2fa3d7693ba9d2e 2026-05-02 07:42:13,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:42:13,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f307cbe2fa3d7693ba9d2e HTTP/1.1" 200 None 2026-05-02 07:42:13,148 - lava_callback - DEBUG - Processing node 69f307cbe2fa3d7693ba9d2e: baseline-arm64-baylibre 2026-05-02 07:42:13,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f307cbe2fa3d7693ba9d2e 2026-05-02 07:42:13,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:42:13,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f307cbe2fa3d7693ba9d2e HTTP/1.1" 200 None 2026-05-02 07:42:13,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:42:13,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f45f5ce4820821ef7fa HTTP/1.1" 200 None 2026-05-02 07:42:13,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:42:13,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f307cbe2fa3d7693ba9d2e HTTP/1.1" 200 None 2026-05-02 07:42:13,201 - lava_callback - ERROR - Error processing callback for node 69f307cbe2fa3d7693ba9d2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56248 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:43:33,375 - lava_callback - INFO - Received callback for node 69f3081be2fa3d7693ba9d59 INFO: 172.20.0.1:39436 - "POST /node/69f3081be2fa3d7693ba9d59 HTTP/1.0" 202 Accepted 2026-05-02 07:43:33,381 - lava_callback - INFO - Starting async job submit for node 69f3081be2fa3d7693ba9d59 2026-05-02 07:43:33,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:43:33,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3081be2fa3d7693ba9d59 HTTP/1.1" 200 None 2026-05-02 07:43:33,396 - lava_callback - DEBUG - Processing node 69f3081be2fa3d7693ba9d59: baseline-arm64-baylibre 2026-05-02 07:43:33,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f3081be2fa3d7693ba9d59 2026-05-02 07:43:33,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:43:33,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3081be2fa3d7693ba9d59 HTTP/1.1" 200 None 2026-05-02 07:43:33,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:43:33,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f45f5ce4820821ef7fa HTTP/1.1" 200 None 2026-05-02 07:43:33,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:43:33,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f3081be2fa3d7693ba9d59 HTTP/1.1" 200 None 2026-05-02 07:43:33,456 - lava_callback - ERROR - Error processing callback for node 69f3081be2fa3d7693ba9d59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:44:53,272 - lava_callback - INFO - Received callback for node 69f30869e2fa3d7693ba9d84 INFO: 172.20.0.1:54158 - "POST /node/69f30869e2fa3d7693ba9d84 HTTP/1.0" 202 Accepted 2026-05-02 07:44:53,278 - lava_callback - INFO - Starting async job submit for node 69f30869e2fa3d7693ba9d84 2026-05-02 07:44:53,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:44:53,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30869e2fa3d7693ba9d84 HTTP/1.1" 200 None 2026-05-02 07:44:53,294 - lava_callback - DEBUG - Processing node 69f30869e2fa3d7693ba9d84: baseline-arm64-baylibre 2026-05-02 07:44:53,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30869e2fa3d7693ba9d84 2026-05-02 07:44:53,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:44:53,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30869e2fa3d7693ba9d84 HTTP/1.1" 200 None 2026-05-02 07:44:53,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:44:53,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f45f5ce4820821ef7fa HTTP/1.1" 200 None 2026-05-02 07:44:53,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:44:53,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30869e2fa3d7693ba9d84 HTTP/1.1" 200 None 2026-05-02 07:44:53,347 - lava_callback - ERROR - Error processing callback for node 69f30869e2fa3d7693ba9d84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59364 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:47:13,360 - lava_callback - INFO - Received callback for node 69f308f1e2fa3d7693ba9dc9 INFO: 172.20.0.1:38770 - "POST /node/69f308f1e2fa3d7693ba9dc9 HTTP/1.0" 202 Accepted 2026-05-02 07:47:13,365 - lava_callback - INFO - Starting async job submit for node 69f308f1e2fa3d7693ba9dc9 2026-05-02 07:47:13,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:13,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f308f1e2fa3d7693ba9dc9 HTTP/1.1" 200 None 2026-05-02 07:47:13,380 - lava_callback - DEBUG - Processing node 69f308f1e2fa3d7693ba9dc9: baseline-arm-baylibre 2026-05-02 07:47:13,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f308f1e2fa3d7693ba9dc9 2026-05-02 07:47:13,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:13,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f308f1e2fa3d7693ba9dc9 HTTP/1.1" 200 None 2026-05-02 07:47:13,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:13,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 07:47:13,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:13,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f308f1e2fa3d7693ba9dc9 HTTP/1.1" 200 None 2026-05-02 07:47:13,430 - lava_callback - ERROR - Error processing callback for node 69f308f1e2fa3d7693ba9dc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37188 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:47:53,400 - lava_callback - INFO - Received callback for node 69f30916e2fa3d7693ba9dd1 INFO: 172.20.0.1:59846 - "POST /node/69f30916e2fa3d7693ba9dd1 HTTP/1.0" 202 Accepted 2026-05-02 07:47:53,405 - lava_callback - INFO - Starting async job submit for node 69f30916e2fa3d7693ba9dd1 2026-05-02 07:47:53,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:53,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30916e2fa3d7693ba9dd1 HTTP/1.1" 200 None 2026-05-02 07:47:53,420 - lava_callback - DEBUG - Processing node 69f30916e2fa3d7693ba9dd1: baseline-arm-baylibre 2026-05-02 07:47:53,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30916e2fa3d7693ba9dd1 2026-05-02 07:47:53,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:53,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30916e2fa3d7693ba9dd1 HTTP/1.1" 200 None 2026-05-02 07:47:53,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:53,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 07:47:53,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:47:53,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30916e2fa3d7693ba9dd1 HTTP/1.1" 200 None 2026-05-02 07:47:53,464 - lava_callback - ERROR - Error processing callback for node 69f30916e2fa3d7693ba9dd1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56080 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:48:13,403 - lava_callback - INFO - Received callback for node 69f3093ae2fa3d7693ba9de3 INFO: 172.20.0.1:56160 - "POST /node/69f3093ae2fa3d7693ba9de3 HTTP/1.0" 202 Accepted 2026-05-02 07:48:13,408 - lava_callback - INFO - Starting async job submit for node 69f3093ae2fa3d7693ba9de3 2026-05-02 07:48:13,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:13,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3093ae2fa3d7693ba9de3 HTTP/1.1" 200 None 2026-05-02 07:48:13,430 - lava_callback - DEBUG - Processing node 69f3093ae2fa3d7693ba9de3: baseline-arm-baylibre 2026-05-02 07:48:13,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f3093ae2fa3d7693ba9de3 2026-05-02 07:48:13,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:13,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3093ae2fa3d7693ba9de3 HTTP/1.1" 200 None 2026-05-02 07:48:13,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:13,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 07:48:13,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:13,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f3093ae2fa3d7693ba9de3 HTTP/1.1" 200 None 2026-05-02 07:48:13,486 - lava_callback - ERROR - Error processing callback for node 69f3093ae2fa3d7693ba9de3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48536 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:48:53,525 - lava_callback - INFO - Received callback for node 69f3095ee2fa3d7693ba9df1 INFO: 172.20.0.1:47160 - "POST /node/69f3095ee2fa3d7693ba9df1 HTTP/1.0" 202 Accepted 2026-05-02 07:48:53,530 - lava_callback - INFO - Starting async job submit for node 69f3095ee2fa3d7693ba9df1 2026-05-02 07:48:53,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:53,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3095ee2fa3d7693ba9df1 HTTP/1.1" 200 None 2026-05-02 07:48:53,545 - lava_callback - DEBUG - Processing node 69f3095ee2fa3d7693ba9df1: baseline-arm-baylibre 2026-05-02 07:48:53,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f3095ee2fa3d7693ba9df1 2026-05-02 07:48:53,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:53,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f3095ee2fa3d7693ba9df1 HTTP/1.1" 200 None 2026-05-02 07:48:53,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:53,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 07:48:53,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:48:53,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f3095ee2fa3d7693ba9df1 HTTP/1.1" 200 None 2026-05-02 07:48:53,592 - lava_callback - ERROR - Error processing callback for node 69f3095ee2fa3d7693ba9df1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32876 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:49:33,496 - lava_callback - INFO - Received callback for node 69f30983e2fa3d7693ba9e03 INFO: 172.20.0.1:39664 - "POST /node/69f30983e2fa3d7693ba9e03 HTTP/1.0" 202 Accepted 2026-05-02 07:49:33,502 - lava_callback - INFO - Starting async job submit for node 69f30983e2fa3d7693ba9e03 2026-05-02 07:49:33,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:49:33,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30983e2fa3d7693ba9e03 HTTP/1.1" 200 None 2026-05-02 07:49:33,517 - lava_callback - DEBUG - Processing node 69f30983e2fa3d7693ba9e03: baseline-arm-baylibre 2026-05-02 07:49:33,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30983e2fa3d7693ba9e03 2026-05-02 07:49:33,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:49:33,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30983e2fa3d7693ba9e03 HTTP/1.1" 200 None 2026-05-02 07:49:33,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:49:33,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 07:49:33,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:49:33,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30983e2fa3d7693ba9e03 HTTP/1.1" 200 None 2026-05-02 07:49:33,562 - lava_callback - ERROR - Error processing callback for node 69f30983e2fa3d7693ba9e03: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51522 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:50:13,542 - lava_callback - INFO - Received callback for node 69f309a8e2fa3d7693ba9e12 INFO: 172.20.0.1:51196 - "POST /node/69f309a8e2fa3d7693ba9e12 HTTP/1.0" 202 Accepted 2026-05-02 07:50:13,548 - lava_callback - INFO - Starting async job submit for node 69f309a8e2fa3d7693ba9e12 2026-05-02 07:50:13,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:50:13,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f309a8e2fa3d7693ba9e12 HTTP/1.1" 200 None 2026-05-02 07:50:13,563 - lava_callback - DEBUG - Processing node 69f309a8e2fa3d7693ba9e12: baseline-arm-baylibre 2026-05-02 07:50:13,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f309a8e2fa3d7693ba9e12 2026-05-02 07:50:13,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:50:13,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f309a8e2fa3d7693ba9e12 HTTP/1.1" 200 None 2026-05-02 07:50:13,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:50:13,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 07:50:13,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:50:13,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f309a8e2fa3d7693ba9e12 HTTP/1.1" 200 None 2026-05-02 07:50:13,624 - lava_callback - ERROR - Error processing callback for node 69f309a8e2fa3d7693ba9e12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33370 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:52:12,158 - lava_callback - INFO - Received callback for node 69f30a16e2fa3d7693ba9e48 INFO: 172.20.0.1:39314 - "POST /node/69f30a16e2fa3d7693ba9e48 HTTP/1.0" 202 Accepted 2026-05-02 07:52:12,163 - lava_callback - INFO - Starting async job submit for node 69f30a16e2fa3d7693ba9e48 2026-05-02 07:52:12,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:52:12,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30a16e2fa3d7693ba9e48 HTTP/1.1" 200 None 2026-05-02 07:52:12,179 - lava_callback - DEBUG - Processing node 69f30a16e2fa3d7693ba9e48: baseline-arm-clabbe 2026-05-02 07:52:12,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30a16e2fa3d7693ba9e48 2026-05-02 07:52:12,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:52:12,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30a16e2fa3d7693ba9e48 HTTP/1.1" 200 None 2026-05-02 07:52:12,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:52:12,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f22f31f5ce4820821ef7eb HTTP/1.1" 200 None 2026-05-02 07:52:12,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:52:12,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30a16e2fa3d7693ba9e48 HTTP/1.1" 200 None 2026-05-02 07:52:12,225 - lava_callback - ERROR - Error processing callback for node 69f30a16e2fa3d7693ba9e48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37286 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:57:53,996 - lava_callback - INFO - Received callback for node 69f30b7de2fa3d7693ba9ee6 INFO: 172.20.0.1:52392 - "POST /node/69f30b7de2fa3d7693ba9ee6 HTTP/1.0" 202 Accepted 2026-05-02 07:57:54,002 - lava_callback - INFO - Starting async job submit for node 69f30b7de2fa3d7693ba9ee6 2026-05-02 07:57:54,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:57:54,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30b7de2fa3d7693ba9ee6 HTTP/1.1" 200 None 2026-05-02 07:57:54,018 - lava_callback - DEBUG - Processing node 69f30b7de2fa3d7693ba9ee6: baseline-arm64-baylibre 2026-05-02 07:57:54,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30b7de2fa3d7693ba9ee6 2026-05-02 07:57:54,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:57:54,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30b7de2fa3d7693ba9ee6 HTTP/1.1" 200 None 2026-05-02 07:57:54,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:57:54,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 07:57:54,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:57:54,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30b7de2fa3d7693ba9ee6 HTTP/1.1" 200 None 2026-05-02 07:57:54,072 - lava_callback - ERROR - Error processing callback for node 69f30b7de2fa3d7693ba9ee6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35154 - "GET / HTTP/1.0" 200 OK 2026-05-02 07:59:14,071 - lava_callback - INFO - Received callback for node 69f30bcee2fa3d7693ba9f14 INFO: 172.20.0.1:37456 - "POST /node/69f30bcee2fa3d7693ba9f14 HTTP/1.0" 202 Accepted 2026-05-02 07:59:14,077 - lava_callback - INFO - Starting async job submit for node 69f30bcee2fa3d7693ba9f14 2026-05-02 07:59:14,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:59:14,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30bcee2fa3d7693ba9f14 HTTP/1.1" 200 None 2026-05-02 07:59:14,092 - lava_callback - DEBUG - Processing node 69f30bcee2fa3d7693ba9f14: baseline-arm64-baylibre 2026-05-02 07:59:14,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30bcee2fa3d7693ba9f14 2026-05-02 07:59:14,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:59:14,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30bcee2fa3d7693ba9f14 HTTP/1.1" 200 None 2026-05-02 07:59:14,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:59:14,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 07:59:14,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 07:59:14,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30bcee2fa3d7693ba9f14 HTTP/1.1" 200 None 2026-05-02 07:59:14,142 - lava_callback - ERROR - Error processing callback for node 69f30bcee2fa3d7693ba9f14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50270 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:00:34,155 - lava_callback - INFO - Received callback for node 69f30c1de2fa3d7693ba9f39 INFO: 172.20.0.1:44466 - "POST /node/69f30c1de2fa3d7693ba9f39 HTTP/1.0" 202 Accepted 2026-05-02 08:00:34,160 - lava_callback - INFO - Starting async job submit for node 69f30c1de2fa3d7693ba9f39 2026-05-02 08:00:34,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:00:34,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30c1de2fa3d7693ba9f39 HTTP/1.1" 200 None 2026-05-02 08:00:34,176 - lava_callback - DEBUG - Processing node 69f30c1de2fa3d7693ba9f39: baseline-arm64-baylibre 2026-05-02 08:00:34,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30c1de2fa3d7693ba9f39 2026-05-02 08:00:34,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:00:34,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30c1de2fa3d7693ba9f39 HTTP/1.1" 200 None 2026-05-02 08:00:34,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:00:34,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:00:34,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:00:34,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30c1de2fa3d7693ba9f39 HTTP/1.1" 200 None 2026-05-02 08:00:34,260 - lava_callback - ERROR - Error processing callback for node 69f30c1de2fa3d7693ba9f39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55652 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:01:54,180 - lava_callback - INFO - Received callback for node 69f30c6be2fa3d7693ba9f62 INFO: 172.20.0.1:44324 - "POST /node/69f30c6be2fa3d7693ba9f62 HTTP/1.0" 202 Accepted 2026-05-02 08:01:54,185 - lava_callback - INFO - Starting async job submit for node 69f30c6be2fa3d7693ba9f62 2026-05-02 08:01:54,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:01:54,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30c6be2fa3d7693ba9f62 HTTP/1.1" 200 None 2026-05-02 08:01:54,200 - lava_callback - DEBUG - Processing node 69f30c6be2fa3d7693ba9f62: baseline-arm64-baylibre 2026-05-02 08:01:54,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30c6be2fa3d7693ba9f62 2026-05-02 08:01:54,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:01:54,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30c6be2fa3d7693ba9f62 HTTP/1.1" 200 None 2026-05-02 08:01:54,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:01:54,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:01:54,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:01:54,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30c6be2fa3d7693ba9f62 HTTP/1.1" 200 None 2026-05-02 08:01:54,244 - lava_callback - ERROR - Error processing callback for node 69f30c6be2fa3d7693ba9f62: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59286 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:04:14,305 - lava_callback - INFO - Received callback for node 69f30cf6e2fa3d7693ba9f92 INFO: 172.20.0.1:34416 - "POST /node/69f30cf6e2fa3d7693ba9f92 HTTP/1.0" 202 Accepted 2026-05-02 08:04:14,311 - lava_callback - INFO - Starting async job submit for node 69f30cf6e2fa3d7693ba9f92 2026-05-02 08:04:14,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:04:14,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30cf6e2fa3d7693ba9f92 HTTP/1.1" 200 None 2026-05-02 08:04:14,326 - lava_callback - DEBUG - Processing node 69f30cf6e2fa3d7693ba9f92: baseline-arm64-baylibre 2026-05-02 08:04:14,326 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30cf6e2fa3d7693ba9f92 2026-05-02 08:04:14,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:04:14,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30cf6e2fa3d7693ba9f92 HTTP/1.1" 200 None 2026-05-02 08:04:14,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:04:14,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:04:14,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:04:14,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30cf6e2fa3d7693ba9f92 HTTP/1.1" 200 None 2026-05-02 08:04:14,378 - lava_callback - ERROR - Error processing callback for node 69f30cf6e2fa3d7693ba9f92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:05:34,353 - lava_callback - INFO - Received callback for node 69f30d48e2fa3d7693ba9fb0 INFO: 172.20.0.1:40226 - "POST /node/69f30d48e2fa3d7693ba9fb0 HTTP/1.0" 202 Accepted 2026-05-02 08:05:34,359 - lava_callback - INFO - Starting async job submit for node 69f30d48e2fa3d7693ba9fb0 2026-05-02 08:05:34,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:05:34,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30d48e2fa3d7693ba9fb0 HTTP/1.1" 200 None 2026-05-02 08:05:34,373 - lava_callback - DEBUG - Processing node 69f30d48e2fa3d7693ba9fb0: baseline-arm64-baylibre 2026-05-02 08:05:34,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30d48e2fa3d7693ba9fb0 2026-05-02 08:05:34,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:05:34,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30d48e2fa3d7693ba9fb0 HTTP/1.1" 200 None 2026-05-02 08:05:34,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:05:34,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:05:34,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:05:34,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30d48e2fa3d7693ba9fb0 HTTP/1.1" 200 None 2026-05-02 08:05:34,419 - lava_callback - ERROR - Error processing callback for node 69f30d48e2fa3d7693ba9fb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:07:12,825 - lava_callback - INFO - Received callback for node 69f30d9fe2fa3d7693ba9fd3 INFO: 172.20.0.1:45670 - "POST /node/69f30d9fe2fa3d7693ba9fd3 HTTP/1.0" 202 Accepted 2026-05-02 08:07:12,830 - lava_callback - INFO - Starting async job submit for node 69f30d9fe2fa3d7693ba9fd3 2026-05-02 08:07:12,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:07:12,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30d9fe2fa3d7693ba9fd3 HTTP/1.1" 200 None 2026-05-02 08:07:12,846 - lava_callback - DEBUG - Processing node 69f30d9fe2fa3d7693ba9fd3: baseline-arm64-clabbe 2026-05-02 08:07:12,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30d9fe2fa3d7693ba9fd3 2026-05-02 08:07:12,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:07:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30d9fe2fa3d7693ba9fd3 HTTP/1.1" 200 None 2026-05-02 08:07:12,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:07:12,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:07:12,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:07:12,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30d9fe2fa3d7693ba9fd3 HTTP/1.1" 200 None 2026-05-02 08:07:12,890 - lava_callback - ERROR - Error processing callback for node 69f30d9fe2fa3d7693ba9fd3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34854 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:08:32,889 - lava_callback - INFO - Received callback for node 69f30df4e2fa3d7693ba9ff8 INFO: 172.20.0.1:43116 - "POST /node/69f30df4e2fa3d7693ba9ff8 HTTP/1.0" 202 Accepted 2026-05-02 08:08:32,894 - lava_callback - INFO - Starting async job submit for node 69f30df4e2fa3d7693ba9ff8 2026-05-02 08:08:32,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:08:32,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30df4e2fa3d7693ba9ff8 HTTP/1.1" 200 None 2026-05-02 08:08:32,913 - lava_callback - DEBUG - Processing node 69f30df4e2fa3d7693ba9ff8: baseline-arm64-clabbe 2026-05-02 08:08:32,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30df4e2fa3d7693ba9ff8 2026-05-02 08:08:32,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:08:32,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30df4e2fa3d7693ba9ff8 HTTP/1.1" 200 None 2026-05-02 08:08:32,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:08:32,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:08:32,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:08:32,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30df4e2fa3d7693ba9ff8 HTTP/1.1" 200 None 2026-05-02 08:08:32,962 - lava_callback - ERROR - Error processing callback for node 69f30df4e2fa3d7693ba9ff8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47902 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:09:52,995 - lava_callback - INFO - Received callback for node 69f30e45e2fa3d7693baa021 INFO: 172.20.0.1:54000 - "POST /node/69f30e45e2fa3d7693baa021 HTTP/1.0" 202 Accepted 2026-05-02 08:09:53,002 - lava_callback - INFO - Starting async job submit for node 69f30e45e2fa3d7693baa021 2026-05-02 08:09:53,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:09:53,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30e45e2fa3d7693baa021 HTTP/1.1" 200 None 2026-05-02 08:09:53,017 - lava_callback - DEBUG - Processing node 69f30e45e2fa3d7693baa021: baseline-arm64-clabbe 2026-05-02 08:09:53,017 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30e45e2fa3d7693baa021 2026-05-02 08:09:53,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:09:53,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30e45e2fa3d7693baa021 HTTP/1.1" 200 None 2026-05-02 08:09:53,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:09:53,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:09:53,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:09:53,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30e45e2fa3d7693baa021 HTTP/1.1" 200 None 2026-05-02 08:09:53,072 - lava_callback - ERROR - Error processing callback for node 69f30e45e2fa3d7693baa021: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60034 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:11:13,022 - lava_callback - INFO - Received callback for node 69f30e9be2fa3d7693baa046 INFO: 172.20.0.1:54960 - "POST /node/69f30e9be2fa3d7693baa046 HTTP/1.0" 202 Accepted 2026-05-02 08:11:13,028 - lava_callback - INFO - Starting async job submit for node 69f30e9be2fa3d7693baa046 2026-05-02 08:11:13,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:11:13,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30e9be2fa3d7693baa046 HTTP/1.1" 200 None 2026-05-02 08:11:13,043 - lava_callback - DEBUG - Processing node 69f30e9be2fa3d7693baa046: baseline-arm64-clabbe 2026-05-02 08:11:13,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30e9be2fa3d7693baa046 2026-05-02 08:11:13,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:11:13,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30e9be2fa3d7693baa046 HTTP/1.1" 200 None 2026-05-02 08:11:13,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:11:13,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:11:13,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:11:13,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30e9be2fa3d7693baa046 HTTP/1.1" 200 None 2026-05-02 08:11:13,097 - lava_callback - ERROR - Error processing callback for node 69f30e9be2fa3d7693baa046: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34976 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:12:53,048 - lava_callback - INFO - Received callback for node 69f30eefe2fa3d7693baa062 INFO: 172.20.0.1:37838 - "POST /node/69f30eefe2fa3d7693baa062 HTTP/1.0" 202 Accepted 2026-05-02 08:12:53,054 - lava_callback - INFO - Starting async job submit for node 69f30eefe2fa3d7693baa062 2026-05-02 08:12:53,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:12:53,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30eefe2fa3d7693baa062 HTTP/1.1" 200 None 2026-05-02 08:12:53,069 - lava_callback - DEBUG - Processing node 69f30eefe2fa3d7693baa062: baseline-arm64-clabbe 2026-05-02 08:12:53,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30eefe2fa3d7693baa062 2026-05-02 08:12:53,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:12:53,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30eefe2fa3d7693baa062 HTTP/1.1" 200 None 2026-05-02 08:12:53,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:12:53,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:12:53,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:12:53,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30eefe2fa3d7693baa062 HTTP/1.1" 200 None 2026-05-02 08:12:53,115 - lava_callback - ERROR - Error processing callback for node 69f30eefe2fa3d7693baa062: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58246 - "GET / HTTP/1.0" 200 OK 2026-05-02 08:14:13,161 - lava_callback - INFO - Received callback for node 69f30f44e2fa3d7693baa083 INFO: 172.20.0.1:58464 - "POST /node/69f30f44e2fa3d7693baa083 HTTP/1.0" 202 Accepted 2026-05-02 08:14:13,167 - lava_callback - INFO - Starting async job submit for node 69f30f44e2fa3d7693baa083 2026-05-02 08:14:13,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:14:13,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30f44e2fa3d7693baa083 HTTP/1.1" 200 None 2026-05-02 08:14:13,188 - lava_callback - DEBUG - Processing node 69f30f44e2fa3d7693baa083: baseline-arm64-clabbe 2026-05-02 08:14:13,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f30f44e2fa3d7693baa083 2026-05-02 08:14:13,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:14:13,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f30f44e2fa3d7693baa083 HTTP/1.1" 200 None 2026-05-02 08:14:13,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:14:13,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f1bf6299260e707730df5f HTTP/1.1" 200 None 2026-05-02 08:14:13,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-02 08:14:13,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f30f44e2fa3d7693baa083 HTTP/1.1" 200 None 2026-05-02 08:14:13,251 - lava_callback - ERROR - Error processing callback for node 69f30f44e2fa3d7693baa083: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]