INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46334 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:19:03,625 - lava_callback - INFO - Received callback for node 69c01664c3d73288b0d05333 2026-03-24 16:19:03,632 - lava_callback - INFO - Starting async job submit for node 69c01664c3d73288b0d05333 INFO: 172.21.0.1:46264 - "POST /node/69c01664c3d73288b0d05333 HTTP/1.0" 202 Accepted 2026-03-24 16:19:03,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:19:03,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01664c3d73288b0d05333 HTTP/1.1" 200 None 2026-03-24 16:19:03,664 - lava_callback - DEBUG - Processing node 69c01664c3d73288b0d05333: baseline-arm64-baylibre 2026-03-24 16:19:03,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01664c3d73288b0d05333 2026-03-24 16:19:03,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:19:03,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01664c3d73288b0d05333 HTTP/1.1" 200 None 2026-03-24 16:19:03,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:19:03,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:19:03,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:19:03,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01664c3d73288b0d05333 HTTP/1.1" 200 None 2026-03-24 16:19:03,761 - lava_callback - ERROR - Error processing callback for node 69c01664c3d73288b0d05333: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45900 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:21:43,690 - lava_callback - INFO - Received callback for node 69c01703c3d73288b0d05340 INFO: 172.21.0.1:32814 - "POST /node/69c01703c3d73288b0d05340 HTTP/1.0" 202 Accepted 2026-03-24 16:21:43,695 - lava_callback - INFO - Starting async job submit for node 69c01703c3d73288b0d05340 2026-03-24 16:21:43,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:21:43,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01703c3d73288b0d05340 HTTP/1.1" 200 None 2026-03-24 16:21:43,712 - lava_callback - DEBUG - Processing node 69c01703c3d73288b0d05340: baseline-arm64-baylibre 2026-03-24 16:21:43,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01703c3d73288b0d05340 2026-03-24 16:21:43,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:21:43,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01703c3d73288b0d05340 HTTP/1.1" 200 None 2026-03-24 16:21:43,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:21:43,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:21:43,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:21:43,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01703c3d73288b0d05340 HTTP/1.1" 200 None 2026-03-24 16:21:43,802 - lava_callback - ERROR - Error processing callback for node 69c01703c3d73288b0d05340: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53224 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:23:15,099 - lava_callback - INFO - Received callback for node 69c0175dc3d73288b0d0534c INFO: 172.21.0.1:51388 - "POST /node/69c0175dc3d73288b0d0534c HTTP/1.0" 202 Accepted 2026-03-24 16:23:15,105 - lava_callback - INFO - Starting async job submit for node 69c0175dc3d73288b0d0534c 2026-03-24 16:23:15,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:23:15,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0175dc3d73288b0d0534c HTTP/1.1" 200 None 2026-03-24 16:23:15,126 - lava_callback - DEBUG - Processing node 69c0175dc3d73288b0d0534c: baseline-arm64-clabbe 2026-03-24 16:23:15,126 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0175dc3d73288b0d0534c 2026-03-24 16:23:15,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:23:15,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0175dc3d73288b0d0534c HTTP/1.1" 200 None 2026-03-24 16:23:15,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:51398 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:23:15,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:23:15,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:23:15,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0175dc3d73288b0d0534c HTTP/1.1" 200 None 2026-03-24 16:23:15,218 - lava_callback - ERROR - Error processing callback for node 69c0175dc3d73288b0d0534c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:24:35,125 - lava_callback - INFO - Received callback for node 69c017aec3d73288b0d0535a INFO: 172.21.0.1:46270 - "POST /node/69c017aec3d73288b0d0535a HTTP/1.0" 202 Accepted 2026-03-24 16:24:35,131 - lava_callback - INFO - Starting async job submit for node 69c017aec3d73288b0d0535a 2026-03-24 16:24:35,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:24:35,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c017aec3d73288b0d0535a HTTP/1.1" 200 None 2026-03-24 16:24:35,147 - lava_callback - DEBUG - Processing node 69c017aec3d73288b0d0535a: baseline-arm64-clabbe 2026-03-24 16:24:35,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c017aec3d73288b0d0535a 2026-03-24 16:24:35,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:24:35,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c017aec3d73288b0d0535a HTTP/1.1" 200 None 2026-03-24 16:24:35,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:24:35,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:24:35,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:24:35,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c017aec3d73288b0d0535a HTTP/1.1" 200 None 2026-03-24 16:24:35,224 - lava_callback - ERROR - Error processing callback for node 69c017aec3d73288b0d0535a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54206 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:25:55,148 - lava_callback - INFO - Received callback for node 69c017fec3d73288b0d05365 INFO: 172.21.0.1:56848 - "POST /node/69c017fec3d73288b0d05365 HTTP/1.0" 202 Accepted 2026-03-24 16:25:55,153 - lava_callback - INFO - Starting async job submit for node 69c017fec3d73288b0d05365 2026-03-24 16:25:55,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:25:55,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c017fec3d73288b0d05365 HTTP/1.1" 200 None 2026-03-24 16:25:55,170 - lava_callback - DEBUG - Processing node 69c017fec3d73288b0d05365: baseline-arm64-clabbe 2026-03-24 16:25:55,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c017fec3d73288b0d05365 2026-03-24 16:25:55,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:25:55,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c017fec3d73288b0d05365 HTTP/1.1" 200 None 2026-03-24 16:25:55,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:25:55,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:25:55,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:25:55,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c017fec3d73288b0d05365 HTTP/1.1" 200 None 2026-03-24 16:25:55,229 - lava_callback - ERROR - Error processing callback for node 69c017fec3d73288b0d05365: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:27:15,247 - lava_callback - INFO - Received callback for node 69c0184ec3d73288b0d05374 INFO: 172.21.0.1:32940 - "POST /node/69c0184ec3d73288b0d05374 HTTP/1.0" 202 Accepted 2026-03-24 16:27:15,253 - lava_callback - INFO - Starting async job submit for node 69c0184ec3d73288b0d05374 2026-03-24 16:27:15,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:27:15,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0184ec3d73288b0d05374 HTTP/1.1" 200 None 2026-03-24 16:27:15,287 - lava_callback - DEBUG - Processing node 69c0184ec3d73288b0d05374: baseline-arm64-clabbe 2026-03-24 16:27:15,287 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0184ec3d73288b0d05374 2026-03-24 16:27:15,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:27:15,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0184ec3d73288b0d05374 HTTP/1.1" 200 None 2026-03-24 16:27:15,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:27:15,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:27:15,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:27:15,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0184ec3d73288b0d05374 HTTP/1.1" 200 None 2026-03-24 16:27:15,378 - lava_callback - ERROR - Error processing callback for node 69c0184ec3d73288b0d05374: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54554 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:28:35,293 - lava_callback - INFO - Received callback for node 69c0189ec3d73288b0d05380 INFO: 172.21.0.1:41320 - "POST /node/69c0189ec3d73288b0d05380 HTTP/1.0" 202 Accepted 2026-03-24 16:28:35,299 - lava_callback - INFO - Starting async job submit for node 69c0189ec3d73288b0d05380 2026-03-24 16:28:35,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:28:35,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0189ec3d73288b0d05380 HTTP/1.1" 200 None 2026-03-24 16:28:35,334 - lava_callback - DEBUG - Processing node 69c0189ec3d73288b0d05380: baseline-arm64-clabbe 2026-03-24 16:28:35,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0189ec3d73288b0d05380 2026-03-24 16:28:35,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:28:35,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0189ec3d73288b0d05380 HTTP/1.1" 200 None 2026-03-24 16:28:35,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:28:35,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:28:35,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:28:35,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0189ec3d73288b0d05380 HTTP/1.1" 200 None 2026-03-24 16:28:35,427 - lava_callback - ERROR - Error processing callback for node 69c0189ec3d73288b0d05380: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36234 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:29:55,394 - lava_callback - INFO - Received callback for node 69c018eec3d73288b0d07a46 INFO: 172.21.0.1:46404 - "POST /node/69c018eec3d73288b0d07a46 HTTP/1.0" 202 Accepted 2026-03-24 16:29:55,400 - lava_callback - INFO - Starting async job submit for node 69c018eec3d73288b0d07a46 2026-03-24 16:29:55,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:29:55,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c018eec3d73288b0d07a46 HTTP/1.1" 200 None 2026-03-24 16:29:55,417 - lava_callback - DEBUG - Processing node 69c018eec3d73288b0d07a46: baseline-arm64-clabbe 2026-03-24 16:29:55,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c018eec3d73288b0d07a46 2026-03-24 16:29:55,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:29:55,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c018eec3d73288b0d07a46 HTTP/1.1" 200 None 2026-03-24 16:29:55,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:29:55,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:29:55,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:29:55,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c018eec3d73288b0d07a46 HTTP/1.1" 200 None 2026-03-24 16:29:55,493 - lava_callback - ERROR - Error processing callback for node 69c018eec3d73288b0d07a46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54120 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:32:45,315 - lava_callback - INFO - Received callback for node 69c2bc326778148209fd33b4 INFO: 172.21.0.1:52124 - "POST /node/69c2bc326778148209fd33b4 HTTP/1.0" 202 Accepted 2026-03-24 16:32:45,324 - lava_callback - INFO - Starting async job submit for node 69c2bc326778148209fd33b4 2026-03-24 16:32:45,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:32:45,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bc326778148209fd33b4 HTTP/1.1" 200 None 2026-03-24 16:32:45,362 - lava_callback - DEBUG - Processing node 69c2bc326778148209fd33b4: baseline-arm 2026-03-24 16:32:45,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:32:45,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:32:45,823 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bc326778148209fd33b4/log.txt.gz 2026-03-24 16:32:45,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:32:46,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:32:46,118 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bc326778148209fd33b4/lava-logs.yaml 2026-03-24 16:32:46,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:32:46,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:32:46,322 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bc326778148209fd33b4/lava_callback.json.gz 2026-03-24 16:32:46,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:32:46,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bc326778148209fd33b4 HTTP/1.1" 200 None 2026-03-24 16:32:46,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:32:46,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9546778148209fd3372 HTTP/1.1" 200 None 2026-03-24 16:32:46,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:32:46,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bc326778148209fd33b4 HTTP/1.1" 200 None 2026-03-24 16:32:46,429 - lava_callback - INFO - Completed processing callback for node 69c2bc326778148209fd33b4 INFO: 172.21.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:33:03,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:33:03,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:37:44,632 - lava_callback - INFO - Received callback for node 69c01acbc3d73288b0d07a7e INFO: 172.21.0.1:55402 - "POST /node/69c01acbc3d73288b0d07a7e HTTP/1.0" 202 Accepted 2026-03-24 16:37:44,637 - lava_callback - INFO - Starting async job submit for node 69c01acbc3d73288b0d07a7e 2026-03-24 16:37:44,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:37:44,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01acbc3d73288b0d07a7e HTTP/1.1" 200 None 2026-03-24 16:37:44,675 - lava_callback - DEBUG - Processing node 69c01acbc3d73288b0d07a7e: baseline-arm-baylibre 2026-03-24 16:37:44,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01acbc3d73288b0d07a7e 2026-03-24 16:37:44,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:37:44,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01acbc3d73288b0d07a7e HTTP/1.1" 200 None 2026-03-24 16:37:44,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:37:44,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:37:44,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:37:44,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01acbc3d73288b0d07a7e HTTP/1.1" 200 None 2026-03-24 16:37:44,744 - lava_callback - ERROR - Error processing callback for node 69c01acbc3d73288b0d07a7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46772 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:38:44,485 - lava_callback - INFO - Received callback for node 69c01b07c3d73288b0d07a8b INFO: 172.21.0.1:40948 - "POST /node/69c01b07c3d73288b0d07a8b HTTP/1.0" 202 Accepted 2026-03-24 16:38:44,491 - lava_callback - INFO - Starting async job submit for node 69c01b07c3d73288b0d07a8b 2026-03-24 16:38:44,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:38:44,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b07c3d73288b0d07a8b HTTP/1.1" 200 None 2026-03-24 16:38:44,508 - lava_callback - DEBUG - Processing node 69c01b07c3d73288b0d07a8b: baseline-arm-baylibre 2026-03-24 16:38:44,508 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01b07c3d73288b0d07a8b 2026-03-24 16:38:44,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:38:44,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b07c3d73288b0d07a8b HTTP/1.1" 200 None 2026-03-24 16:38:44,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:38:44,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:38:44,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:38:44,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01b07c3d73288b0d07a8b HTTP/1.1" 200 None 2026-03-24 16:38:44,564 - lava_callback - ERROR - Error processing callback for node 69c01b07c3d73288b0d07a8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:39:24,564 - lava_callback - INFO - Received callback for node 69c01b2dc3d73288b0d07a99 INFO: 172.21.0.1:41830 - "POST /node/69c01b2dc3d73288b0d07a99 HTTP/1.0" 202 Accepted 2026-03-24 16:39:24,573 - lava_callback - INFO - Starting async job submit for node 69c01b2dc3d73288b0d07a99 2026-03-24 16:39:24,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:39:24,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b2dc3d73288b0d07a99 HTTP/1.1" 200 None 2026-03-24 16:39:24,602 - lava_callback - DEBUG - Processing node 69c01b2dc3d73288b0d07a99: baseline-arm-baylibre 2026-03-24 16:39:24,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01b2dc3d73288b0d07a99 2026-03-24 16:39:24,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:39:24,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b2dc3d73288b0d07a99 HTTP/1.1" 200 None 2026-03-24 16:39:24,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:39:24,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:39:24,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:39:24,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01b2dc3d73288b0d07a99 HTTP/1.1" 200 None 2026-03-24 16:39:24,706 - lava_callback - ERROR - Error processing callback for node 69c01b2dc3d73288b0d07a99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48168 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:40:04,566 - lava_callback - INFO - Received callback for node 69c01b51c3d73288b0d07aa8 INFO: 172.21.0.1:37522 - "POST /node/69c01b51c3d73288b0d07aa8 HTTP/1.0" 202 Accepted 2026-03-24 16:40:04,577 - lava_callback - INFO - Starting async job submit for node 69c01b51c3d73288b0d07aa8 2026-03-24 16:40:04,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:04,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b51c3d73288b0d07aa8 HTTP/1.1" 200 None 2026-03-24 16:40:04,612 - lava_callback - DEBUG - Processing node 69c01b51c3d73288b0d07aa8: baseline-arm-baylibre 2026-03-24 16:40:04,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01b51c3d73288b0d07aa8 2026-03-24 16:40:04,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:04,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b51c3d73288b0d07aa8 HTTP/1.1" 200 None 2026-03-24 16:40:04,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:04,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:40:04,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:04,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01b51c3d73288b0d07aa8 HTTP/1.1" 200 None 2026-03-24 16:40:04,709 - lava_callback - ERROR - Error processing callback for node 69c01b51c3d73288b0d07aa8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55510 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:40:24,590 - lava_callback - INFO - Received callback for node 69c01b75c3d73288b0d07ab3 INFO: 172.21.0.1:55522 - "POST /node/69c01b75c3d73288b0d07ab3 HTTP/1.0" 202 Accepted 2026-03-24 16:40:24,603 - lava_callback - INFO - Starting async job submit for node 69c01b75c3d73288b0d07ab3 2026-03-24 16:40:24,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:24,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b75c3d73288b0d07ab3 HTTP/1.1" 200 None 2026-03-24 16:40:24,630 - lava_callback - DEBUG - Processing node 69c01b75c3d73288b0d07ab3: baseline-arm-baylibre 2026-03-24 16:40:24,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01b75c3d73288b0d07ab3 2026-03-24 16:40:24,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:24,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b75c3d73288b0d07ab3 HTTP/1.1" 200 None 2026-03-24 16:40:24,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:24,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:40:24,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:40:24,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01b75c3d73288b0d07ab3 HTTP/1.1" 200 None 2026-03-24 16:40:24,817 - lava_callback - ERROR - Error processing callback for node 69c01b75c3d73288b0d07ab3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40796 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:41:04,664 - lava_callback - INFO - Received callback for node 69c01b98c3d73288b0d07ac0 INFO: 172.21.0.1:39700 - "POST /node/69c01b98c3d73288b0d07ac0 HTTP/1.0" 202 Accepted 2026-03-24 16:41:04,672 - lava_callback - INFO - Starting async job submit for node 69c01b98c3d73288b0d07ac0 2026-03-24 16:41:04,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:04,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b98c3d73288b0d07ac0 HTTP/1.1" 200 None 2026-03-24 16:41:04,697 - lava_callback - DEBUG - Processing node 69c01b98c3d73288b0d07ac0: baseline-arm-baylibre 2026-03-24 16:41:04,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01b98c3d73288b0d07ac0 2026-03-24 16:41:04,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:04,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01b98c3d73288b0d07ac0 HTTP/1.1" 200 None 2026-03-24 16:41:04,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:04,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:41:04,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:04,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01b98c3d73288b0d07ac0 HTTP/1.1" 200 None 2026-03-24 16:41:04,778 - lava_callback - ERROR - Error processing callback for node 69c01b98c3d73288b0d07ac0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33316 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:41:41,000 - lava_callback - INFO - Received callback for node 69c2beac6778148209fd340c INFO: 172.21.0.1:56632 - "POST /node/69c2beac6778148209fd340c HTTP/1.0" 202 Accepted 2026-03-24 16:41:41,007 - lava_callback - INFO - Starting async job submit for node 69c2beac6778148209fd340c 2026-03-24 16:41:41,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:41,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beac6778148209fd340c HTTP/1.1" 200 None 2026-03-24 16:41:41,024 - lava_callback - DEBUG - Processing node 69c2beac6778148209fd340c: baseline-arm 2026-03-24 16:41:41,029 - lava_callback - WARNING - Failed to upload log 2026-03-24 16:41:41,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:41:41,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:41:41,166 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beac6778148209fd340c/lava-logs.yaml 2026-03-24 16:41:41,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:41:41,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:41:41,295 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beac6778148209fd340c/lava_callback.json.gz 2026-03-24 16:41:41,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:41,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beac6778148209fd340c HTTP/1.1" 200 None 2026-03-24 16:41:41,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:41,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:41:41,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:41,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beac6778148209fd340c HTTP/1.1" 200 None 2026-03-24 16:41:41,388 - lava_callback - INFO - Completed processing callback for node 69c2beac6778148209fd340c 2026-03-24 16:41:44,623 - lava_callback - INFO - Received callback for node 69c01bbdc3d73288b0d07acf INFO: 172.21.0.1:56636 - "POST /node/69c01bbdc3d73288b0d07acf HTTP/1.0" 202 Accepted 2026-03-24 16:41:44,632 - lava_callback - INFO - Starting async job submit for node 69c01bbdc3d73288b0d07acf 2026-03-24 16:41:44,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:44,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01bbdc3d73288b0d07acf HTTP/1.1" 200 None 2026-03-24 16:41:44,649 - lava_callback - DEBUG - Processing node 69c01bbdc3d73288b0d07acf: baseline-arm-baylibre 2026-03-24 16:41:44,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01bbdc3d73288b0d07acf 2026-03-24 16:41:44,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:44,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01bbdc3d73288b0d07acf HTTP/1.1" 200 None 2026-03-24 16:41:44,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:44,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:41:44,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:41:44,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01bbdc3d73288b0d07acf HTTP/1.1" 200 None 2026-03-24 16:41:44,706 - lava_callback - ERROR - Error processing callback for node 69c01bbdc3d73288b0d07acf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34084 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:42:01,738 - lava_callback - INFO - Received callback for node 69c2beb06778148209fd3418 INFO: 172.21.0.1:51492 - "POST /node/69c2beb06778148209fd3418 HTTP/1.0" 202 Accepted 2026-03-24 16:42:01,790 - lava_callback - INFO - Starting async job submit for node 69c2beb06778148209fd3418 2026-03-24 16:42:01,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:01,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb06778148209fd3418 HTTP/1.1" 200 None 2026-03-24 16:42:01,824 - lava_callback - DEBUG - Processing node 69c2beb06778148209fd3418: baseline-arm 2026-03-24 16:42:01,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:02,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:02,065 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beb06778148209fd3418/log.txt.gz 2026-03-24 16:42:02,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:02,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:02,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beb06778148209fd3418/lava-logs.yaml 2026-03-24 16:42:02,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:02,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:02,444 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beb06778148209fd3418/lava_callback.json.gz 2026-03-24 16:42:02,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:02,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beb06778148209fd3418 HTTP/1.1" 200 None 2026-03-24 16:42:02,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:02,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:42:02,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:02,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beb06778148209fd3418 HTTP/1.1" 200 None 2026-03-24 16:42:02,539 - lava_callback - INFO - Completed processing callback for node 69c2beb06778148209fd3418 2026-03-24 16:42:03,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:03,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40576 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:42:15,716 - lava_callback - INFO - Received callback for node 69c2bead6778148209fd340f INFO: 172.21.0.1:40590 - "POST /node/69c2bead6778148209fd340f HTTP/1.0" 202 Accepted 2026-03-24 16:42:15,766 - lava_callback - INFO - Starting async job submit for node 69c2bead6778148209fd340f 2026-03-24 16:42:15,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:15,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bead6778148209fd340f HTTP/1.1" 200 None 2026-03-24 16:42:15,807 - lava_callback - DEBUG - Processing node 69c2bead6778148209fd340f: baseline-arm 2026-03-24 16:42:15,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:16,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:16,070 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bead6778148209fd340f/log.txt.gz 2026-03-24 16:42:16,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:16,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:16,391 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bead6778148209fd340f/lava-logs.yaml 2026-03-24 16:42:16,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40594 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:42:16,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:16,537 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bead6778148209fd340f/lava_callback.json.gz 2026-03-24 16:42:16,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:16,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bead6778148209fd340f HTTP/1.1" 200 None 2026-03-24 16:42:16,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:16,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:42:16,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:16,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bead6778148209fd340f HTTP/1.1" 200 None 2026-03-24 16:42:16,612 - lava_callback - INFO - Completed processing callback for node 69c2bead6778148209fd340f INFO: 172.21.0.1:56634 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:42:24,689 - lava_callback - INFO - Received callback for node 69c01be9c3d73288b0d07ae1 INFO: 172.21.0.1:56650 - "POST /node/69c01be9c3d73288b0d07ae1 HTTP/1.0" 202 Accepted 2026-03-24 16:42:24,694 - lava_callback - INFO - Starting async job submit for node 69c01be9c3d73288b0d07ae1 2026-03-24 16:42:24,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:24,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01be9c3d73288b0d07ae1 HTTP/1.1" 200 None 2026-03-24 16:42:24,712 - lava_callback - DEBUG - Processing node 69c01be9c3d73288b0d07ae1: baseline-arm64-baylibre 2026-03-24 16:42:24,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01be9c3d73288b0d07ae1 2026-03-24 16:42:24,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:24,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01be9c3d73288b0d07ae1 HTTP/1.1" 200 None 2026-03-24 16:42:24,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:24,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:42:24,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:24,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01be9c3d73288b0d07ae1 HTTP/1.1" 200 None 2026-03-24 16:42:24,772 - lava_callback - ERROR - Error processing callback for node 69c01be9c3d73288b0d07ae1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56652 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:42:32,373 - lava_callback - INFO - Received callback for node 69c2beaa6778148209fd3400 INFO: 172.21.0.1:33818 - "POST /node/69c2beaa6778148209fd3400 HTTP/1.0" 202 Accepted 2026-03-24 16:42:32,379 - lava_callback - INFO - Starting async job submit for node 69c2beaa6778148209fd3400 2026-03-24 16:42:32,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:32,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beaa6778148209fd3400 HTTP/1.1" 200 None 2026-03-24 16:42:32,422 - lava_callback - DEBUG - Processing node 69c2beaa6778148209fd3400: baseline-arm 2026-03-24 16:42:32,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:32,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:32,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beaa6778148209fd3400/log.txt.gz 2026-03-24 16:42:32,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:33,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:33,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beaa6778148209fd3400/lava-logs.yaml 2026-03-24 16:42:33,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:33,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:33,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beaa6778148209fd3400/lava_callback.json.gz 2026-03-24 16:42:33,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:33,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beaa6778148209fd3400 HTTP/1.1" 200 None 2026-03-24 16:42:33,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:33,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:42:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:33,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beaa6778148209fd3400 HTTP/1.1" 200 None 2026-03-24 16:42:33,390 - lava_callback - INFO - Completed processing callback for node 69c2beaa6778148209fd3400 2026-03-24 16:42:33,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:33,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:42:48,604 - lava_callback - INFO - Received callback for node 69c2beaf6778148209fd3415 INFO: 172.21.0.1:51458 - "POST /node/69c2beaf6778148209fd3415 HTTP/1.0" 202 Accepted 2026-03-24 16:42:48,611 - lava_callback - INFO - Starting async job submit for node 69c2beaf6778148209fd3415 2026-03-24 16:42:48,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:48,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beaf6778148209fd3415 HTTP/1.1" 200 None 2026-03-24 16:42:48,651 - lava_callback - DEBUG - Processing node 69c2beaf6778148209fd3415: baseline-arm 2026-03-24 16:42:48,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:48,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:48,983 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beaf6778148209fd3415/log.txt.gz 2026-03-24 16:42:49,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:49,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:49,306 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beaf6778148209fd3415/lava-logs.yaml 2026-03-24 16:42:49,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:49,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:49,437 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2beaf6778148209fd3415/lava_callback.json.gz 2026-03-24 16:42:49,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:49,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beaf6778148209fd3415 HTTP/1.1" 200 None 2026-03-24 16:42:49,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:49,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:42:49,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:49,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beaf6778148209fd3415 HTTP/1.1" 200 None 2026-03-24 16:42:49,540 - lava_callback - INFO - Completed processing callback for node 69c2beaf6778148209fd3415 INFO: 172.21.0.1:53798 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:42:59,449 - lava_callback - INFO - Received callback for node 69c2bed46778148209fd34a6 INFO: 172.21.0.1:53800 - "POST /node/69c2bed46778148209fd34a6 HTTP/1.0" 202 Accepted 2026-03-24 16:42:59,501 - lava_callback - INFO - Starting async job submit for node 69c2bed46778148209fd34a6 2026-03-24 16:42:59,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:42:59,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed46778148209fd34a6 HTTP/1.1" 200 None 2026-03-24 16:42:59,538 - lava_callback - DEBUG - Processing node 69c2bed46778148209fd34a6: baseline-arm-pengutronix 2026-03-24 16:42:59,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:59,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:42:59,772 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed46778148209fd34a6/log.txt.gz 2026-03-24 16:42:59,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:42:59,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:00,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed46778148209fd34a6/lava-logs.yaml 2026-03-24 16:43:00,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:00,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:00,120 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed46778148209fd34a6/lava_callback.json.gz 2026-03-24 16:43:00,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:00,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed46778148209fd34a6 HTTP/1.1" 200 None 2026-03-24 16:43:00,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:00,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:43:00,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:00,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bed46778148209fd34a6 HTTP/1.1" 200 None 2026-03-24 16:43:00,198 - lava_callback - INFO - Completed processing callback for node 69c2bed46778148209fd34a6 2026-03-24 16:43:02,904 - lava_callback - INFO - Received callback for node 69c2bed16778148209fd34a0 INFO: 172.21.0.1:60242 - "POST /node/69c2bed16778148209fd34a0 HTTP/1.0" 202 Accepted 2026-03-24 16:43:02,910 - lava_callback - INFO - Starting async job submit for node 69c2bed16778148209fd34a0 2026-03-24 16:43:02,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:02,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed16778148209fd34a0 HTTP/1.1" 200 None 2026-03-24 16:43:02,948 - lava_callback - DEBUG - Processing node 69c2bed16778148209fd34a0: baseline-arm-pengutronix 2026-03-24 16:43:03,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:03,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:03,248 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed16778148209fd34a0/log.txt.gz 2026-03-24 16:43:03,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:03,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:03,553 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed16778148209fd34a0/lava-logs.yaml 2026-03-24 16:43:03,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:03,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:03,710 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed16778148209fd34a0/lava_callback.json.gz 2026-03-24 16:43:03,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:03,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed16778148209fd34a0 HTTP/1.1" 200 None 2026-03-24 16:43:03,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:03,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:43:03,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:03,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bed16778148209fd34a0 HTTP/1.1" 200 None 2026-03-24 16:43:03,800 - lava_callback - INFO - Completed processing callback for node 69c2bed16778148209fd34a0 2026-03-24 16:43:03,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:03,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 16:43:05,739 - lava_callback - INFO - Received callback for node 69c2bebf6778148209fd343e INFO: 172.21.0.1:60250 - "POST /node/69c2bebf6778148209fd343e HTTP/1.0" 202 Accepted 2026-03-24 16:43:05,745 - lava_callback - INFO - Starting async job submit for node 69c2bebf6778148209fd343e 2026-03-24 16:43:05,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:05,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bebf6778148209fd343e HTTP/1.1" 200 None 2026-03-24 16:43:05,793 - lava_callback - DEBUG - Processing node 69c2bebf6778148209fd343e: baseline-arm-broonie 2026-03-24 16:43:06,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:06,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:06,127 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bebf6778148209fd343e/log.txt.gz 2026-03-24 16:43:06,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:06,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:06,420 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bebf6778148209fd343e/lava-logs.yaml 2026-03-24 16:43:06,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:60256 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:43:06,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:06,556 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bebf6778148209fd343e/lava_callback.json.gz 2026-03-24 16:43:06,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:06,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bebf6778148209fd343e HTTP/1.1" 200 None 2026-03-24 16:43:06,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:06,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:43:06,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:06,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bebf6778148209fd343e HTTP/1.1" 200 None 2026-03-24 16:43:06,640 - lava_callback - INFO - Completed processing callback for node 69c2bebf6778148209fd343e 2026-03-24 16:43:07,481 - lava_callback - INFO - Received callback for node 69c2bec16778148209fd3441 INFO: 172.21.0.1:60264 - "POST /node/69c2bec16778148209fd3441 HTTP/1.0" 202 Accepted 2026-03-24 16:43:07,487 - lava_callback - INFO - Starting async job submit for node 69c2bec16778148209fd3441 2026-03-24 16:43:07,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:07,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec16778148209fd3441 HTTP/1.1" 200 None 2026-03-24 16:43:07,526 - lava_callback - DEBUG - Processing node 69c2bec16778148209fd3441: baseline-arm-broonie 2026-03-24 16:43:07,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:07,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:07,852 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bec16778148209fd3441/log.txt.gz 2026-03-24 16:43:07,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:08,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:08,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bec16778148209fd3441/lava-logs.yaml 2026-03-24 16:43:08,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:08,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:08,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bec16778148209fd3441/lava_callback.json.gz 2026-03-24 16:43:08,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:08,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec16778148209fd3441 HTTP/1.1" 200 None 2026-03-24 16:43:08,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:08,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:43:08,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:08,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bec16778148209fd3441 HTTP/1.1" 200 None 2026-03-24 16:43:08,374 - lava_callback - INFO - Completed processing callback for node 69c2bec16778148209fd3441 INFO: 172.21.0.1:35594 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:43:15,818 - lava_callback - INFO - Received callback for node 69c2bec66778148209fd3455 INFO: 172.21.0.1:35606 - "POST /node/69c2bec66778148209fd3455 HTTP/1.0" 202 Accepted 2026-03-24 16:43:15,824 - lava_callback - INFO - Starting async job submit for node 69c2bec66778148209fd3455 2026-03-24 16:43:15,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:15,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec66778148209fd3455 HTTP/1.1" 200 None 2026-03-24 16:43:15,869 - lava_callback - DEBUG - Processing node 69c2bec66778148209fd3455: baseline-arm-clabbe 2026-03-24 16:43:16,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:16,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:16,191 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2bec66778148209fd3455/log.txt.gz 2026-03-24 16:43:16,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:16,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:16,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2bec66778148209fd3455/lava-logs.yaml INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:43:16,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:16,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:16,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2bec66778148209fd3455/lava_callback.json.gz 2026-03-24 16:43:16,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:16,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec66778148209fd3455 HTTP/1.1" 200 None 2026-03-24 16:43:16,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:16,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:43:16,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:16,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bec66778148209fd3455 HTTP/1.1" 200 None 2026-03-24 16:43:16,780 - lava_callback - INFO - Completed processing callback for node 69c2bec66778148209fd3455 2026-03-24 16:43:17,335 - lava_callback - INFO - Received callback for node 69c2bec26778148209fd3444 INFO: 172.21.0.1:35636 - "POST /node/69c2bec26778148209fd3444 HTTP/1.0" 202 Accepted 2026-03-24 16:43:17,341 - lava_callback - INFO - Starting async job submit for node 69c2bec26778148209fd3444 2026-03-24 16:43:17,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:17,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec26778148209fd3444 HTTP/1.1" 200 None 2026-03-24 16:43:17,379 - lava_callback - DEBUG - Processing node 69c2bec26778148209fd3444: baseline-arm-broonie 2026-03-24 16:43:17,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:17,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:17,725 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bec26778148209fd3444/log.txt.gz 2026-03-24 16:43:17,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:18,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:18,066 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bec26778148209fd3444/lava-logs.yaml 2026-03-24 16:43:18,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:18,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:18,232 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bec26778148209fd3444/lava_callback.json.gz 2026-03-24 16:43:18,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:18,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec26778148209fd3444 HTTP/1.1" 200 None 2026-03-24 16:43:18,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:18,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:43:18,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:18,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bec26778148209fd3444 HTTP/1.1" 200 None 2026-03-24 16:43:18,327 - lava_callback - INFO - Completed processing callback for node 69c2bec26778148209fd3444 INFO: 172.21.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43380 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:43:33,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:33,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:48552 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:43:41,133 - lava_callback - INFO - Received callback for node 69c2bf326778148209fd3550 INFO: 172.21.0.1:33378 - "POST /node/69c2bf326778148209fd3550 HTTP/1.0" 202 Accepted 2026-03-24 16:43:41,139 - lava_callback - INFO - Starting async job submit for node 69c2bf326778148209fd3550 2026-03-24 16:43:41,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:41,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bf326778148209fd3550 HTTP/1.1" 200 None 2026-03-24 16:43:41,157 - lava_callback - DEBUG - Processing node 69c2bf326778148209fd3550: baseline-arm 2026-03-24 16:43:41,162 - lava_callback - WARNING - Failed to upload log 2026-03-24 16:43:41,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:41,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:41,325 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bf326778148209fd3550/lava-logs.yaml 2026-03-24 16:43:41,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:43:41,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:43:41,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bf326778148209fd3550/lava_callback.json.gz 2026-03-24 16:43:41,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:41,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bf326778148209fd3550 HTTP/1.1" 200 None 2026-03-24 16:43:41,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:41,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:43:41,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:41,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bf326778148209fd3550 HTTP/1.1" 200 None 2026-03-24 16:43:41,519 - lava_callback - INFO - Completed processing callback for node 69c2bf326778148209fd3550 2026-03-24 16:43:44,824 - lava_callback - INFO - Received callback for node 69c01c2cc3d73288b0d07afc INFO: 172.21.0.1:33394 - "POST /node/69c01c2cc3d73288b0d07afc HTTP/1.0" 202 Accepted 2026-03-24 16:43:44,829 - lava_callback - INFO - Starting async job submit for node 69c01c2cc3d73288b0d07afc 2026-03-24 16:43:44,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:44,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c2cc3d73288b0d07afc HTTP/1.1" 200 None 2026-03-24 16:43:44,845 - lava_callback - DEBUG - Processing node 69c01c2cc3d73288b0d07afc: baseline-arm-baylibre 2026-03-24 16:43:44,845 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c2cc3d73288b0d07afc 2026-03-24 16:43:44,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:44,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c2cc3d73288b0d07afc HTTP/1.1" 200 None 2026-03-24 16:43:44,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:44,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:43:44,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:43:44,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c2cc3d73288b0d07afc HTTP/1.1" 200 None 2026-03-24 16:43:44,911 - lava_callback - ERROR - Error processing callback for node 69c01c2cc3d73288b0d07afc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:44:01,651 - lava_callback - INFO - Received callback for node 69c2bed06778148209fd349d INFO: 172.21.0.1:52750 - "POST /node/69c2bed06778148209fd349d HTTP/1.0" 202 Accepted 2026-03-24 16:44:01,657 - lava_callback - INFO - Starting async job submit for node 69c2bed06778148209fd349d 2026-03-24 16:44:01,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:01,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed06778148209fd349d HTTP/1.1" 200 None 2026-03-24 16:44:01,694 - lava_callback - DEBUG - Processing node 69c2bed06778148209fd349d: baseline-arm-pengutronix 2026-03-24 16:44:01,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:44:02,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:44:02,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed06778148209fd349d/log.txt.gz 2026-03-24 16:44:02,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:44:02,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:44:02,259 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed06778148209fd349d/lava-logs.yaml 2026-03-24 16:44:02,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:44:02,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:44:02,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed06778148209fd349d/lava_callback.json.gz 2026-03-24 16:44:02,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:02,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed06778148209fd349d HTTP/1.1" 200 None 2026-03-24 16:44:02,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:02,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:44:02,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:02,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bed06778148209fd349d HTTP/1.1" 200 None 2026-03-24 16:44:02,561 - lava_callback - INFO - Completed processing callback for node 69c2bed06778148209fd349d 2026-03-24 16:44:03,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:03,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60304 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:44:15,817 - lava_callback - INFO - Received callback for node 69c01c52c3d73288b0d07b1d INFO: 172.21.0.1:60310 - "POST /node/69c01c52c3d73288b0d07b1d HTTP/1.0" 202 Accepted 2026-03-24 16:44:15,823 - lava_callback - INFO - Starting async job submit for node 69c01c52c3d73288b0d07b1d 2026-03-24 16:44:15,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:15,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c52c3d73288b0d07b1d HTTP/1.1" 200 None 2026-03-24 16:44:15,845 - lava_callback - DEBUG - Processing node 69c01c52c3d73288b0d07b1d: baseline-arm-clabbe 2026-03-24 16:44:15,845 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c52c3d73288b0d07b1d 2026-03-24 16:44:15,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:15,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c52c3d73288b0d07b1d HTTP/1.1" 200 None 2026-03-24 16:44:15,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:15,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:44:15,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:15,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c52c3d73288b0d07b1d HTTP/1.1" 200 None 2026-03-24 16:44:15,908 - lava_callback - ERROR - Error processing callback for node 69c01c52c3d73288b0d07b1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:44:24,537 - lava_callback - INFO - Received callback for node 69c2beef6778148209fd34d9 INFO: 172.21.0.1:36602 - "POST /node/69c2beef6778148209fd34d9 HTTP/1.0" 202 Accepted 2026-03-24 16:44:24,543 - lava_callback - INFO - Starting async job submit for node 69c2beef6778148209fd34d9 2026-03-24 16:44:24,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:24,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beef6778148209fd34d9 HTTP/1.1" 200 None 2026-03-24 16:44:24,582 - lava_callback - DEBUG - Processing node 69c2beef6778148209fd34d9: ltp-dio 2026-03-24 16:44:24,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:44:25,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:44:25,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c2beef6778148209fd34d9/log.txt.gz 2026-03-24 16:44:25,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:44:25,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:44:25,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c2beef6778148209fd34d9/lava-logs.yaml 2026-03-24 16:44:25,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:44:25,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:44:25,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c2beef6778148209fd34d9/lava_callback.json.gz 2026-03-24 16:44:25,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:25,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beef6778148209fd34d9 HTTP/1.1" 200 None 2026-03-24 16:44:25,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:25,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:44:25,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:25,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beef6778148209fd34d9 HTTP/1.1" 200 None 2026-03-24 16:44:25,885 - lava_callback - INFO - Completed processing callback for node 69c2beef6778148209fd34d9 INFO: 172.21.0.1:36618 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:44:33,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:33,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56910 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:44:55,857 - lava_callback - INFO - Received callback for node 69c01c77c3d73288b0d07b40 INFO: 172.21.0.1:34350 - "POST /node/69c01c77c3d73288b0d07b40 HTTP/1.0" 202 Accepted 2026-03-24 16:44:55,862 - lava_callback - INFO - Starting async job submit for node 69c01c77c3d73288b0d07b40 2026-03-24 16:44:55,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:55,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c77c3d73288b0d07b40 HTTP/1.1" 200 None 2026-03-24 16:44:55,878 - lava_callback - DEBUG - Processing node 69c01c77c3d73288b0d07b40: baseline-arm-clabbe 2026-03-24 16:44:55,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c77c3d73288b0d07b40 2026-03-24 16:44:55,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:55,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c77c3d73288b0d07b40 HTTP/1.1" 200 None 2026-03-24 16:44:55,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:55,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:44:55,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:44:55,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c77c3d73288b0d07b40 HTTP/1.1" 200 None 2026-03-24 16:44:55,945 - lava_callback - ERROR - Error processing callback for node 69c01c77c3d73288b0d07b40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34352 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:45:02,821 - lava_callback - INFO - Received callback for node 69c2bec76778148209fd3459 INFO: 172.21.0.1:54272 - "POST /node/69c2bec76778148209fd3459 HTTP/1.0" 202 Accepted 2026-03-24 16:45:02,831 - lava_callback - INFO - Starting async job submit for node 69c2bec76778148209fd3459 2026-03-24 16:45:02,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:02,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec76778148209fd3459 HTTP/1.1" 200 None 2026-03-24 16:45:02,914 - lava_callback - DEBUG - Processing node 69c2bec76778148209fd3459: baseline-arm-clabbe 2026-03-24 16:45:03,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:45:03,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:45:03,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2bec76778148209fd3459/log.txt.gz 2026-03-24 16:45:03,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:45:04,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:45:04,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2bec76778148209fd3459/lava-logs.yaml 2026-03-24 16:45:04,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:45:04,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:45:04,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2bec76778148209fd3459/lava_callback.json.gz 2026-03-24 16:45:04,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:04,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bec76778148209fd3459 HTTP/1.1" 200 None 2026-03-24 16:45:04,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:04,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:45:04,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:04,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bec76778148209fd3459 HTTP/1.1" 200 None 2026-03-24 16:45:04,805 - lava_callback - INFO - Completed processing callback for node 69c2bec76778148209fd3459 2026-03-24 16:45:04,968 - lava_callback - INFO - Received callback for node 69c01c8cc3d73288b0d07b5f INFO: 172.21.0.1:54280 - "POST /node/69c01c8cc3d73288b0d07b5f HTTP/1.0" 202 Accepted 2026-03-24 16:45:04,981 - lava_callback - INFO - Starting async job submit for node 69c01c8cc3d73288b0d07b5f 2026-03-24 16:45:04,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8cc3d73288b0d07b5f HTTP/1.1" 200 None 2026-03-24 16:45:05,017 - lava_callback - DEBUG - Processing node 69c01c8cc3d73288b0d07b5f: baseline-arm-baylibre 2026-03-24 16:45:05,017 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c8cc3d73288b0d07b5f 2026-03-24 16:45:05,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8cc3d73288b0d07b5f HTTP/1.1" 200 None 2026-03-24 16:45:05,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:05,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c8cc3d73288b0d07b5f HTTP/1.1" 200 None 2026-03-24 16:45:05,138 - lava_callback - ERROR - Error processing callback for node 69c01c8cc3d73288b0d07b5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:05,624 - lava_callback - INFO - Received callback for node 69c01c8ec3d73288b0d07b62 INFO: 172.21.0.1:54294 - "POST /node/69c01c8ec3d73288b0d07b62 HTTP/1.0" 202 Accepted 2026-03-24 16:45:05,633 - lava_callback - INFO - Starting async job submit for node 69c01c8ec3d73288b0d07b62 2026-03-24 16:45:05,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8ec3d73288b0d07b62 HTTP/1.1" 200 None 2026-03-24 16:45:05,662 - lava_callback - DEBUG - Processing node 69c01c8ec3d73288b0d07b62: baseline-arm-baylibre 2026-03-24 16:45:05,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c8ec3d73288b0d07b62 2026-03-24 16:45:05,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8ec3d73288b0d07b62 HTTP/1.1" 200 None 2026-03-24 16:45:05,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:05,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:05,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c8ec3d73288b0d07b62 HTTP/1.1" 200 None 2026-03-24 16:45:05,757 - lava_callback - ERROR - Error processing callback for node 69c01c8ec3d73288b0d07b62: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43810 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:45:24,814 - lava_callback - INFO - Received callback for node 69c01c8fc3d73288b0d07b67 INFO: 172.21.0.1:43818 - "POST /node/69c01c8fc3d73288b0d07b67 HTTP/1.0" 202 Accepted 2026-03-24 16:45:24,822 - lava_callback - INFO - Starting async job submit for node 69c01c8fc3d73288b0d07b67 2026-03-24 16:45:24,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:24,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8fc3d73288b0d07b67 HTTP/1.1" 200 None 2026-03-24 16:45:24,861 - lava_callback - DEBUG - Processing node 69c01c8fc3d73288b0d07b67: baseline-arm-baylibre 2026-03-24 16:45:24,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c8fc3d73288b0d07b67 2026-03-24 16:45:24,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:24,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8fc3d73288b0d07b67 HTTP/1.1" 200 None 2026-03-24 16:45:24,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:24,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:24,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:24,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c8fc3d73288b0d07b67 HTTP/1.1" 200 None 2026-03-24 16:45:24,961 - lava_callback - ERROR - Error processing callback for node 69c01c8fc3d73288b0d07b67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:25,473 - lava_callback - INFO - Received callback for node 69c01c93c3d73288b0d07b86 INFO: 172.21.0.1:43832 - "POST /node/69c01c93c3d73288b0d07b86 HTTP/1.0" 202 Accepted 2026-03-24 16:45:25,481 - lava_callback - INFO - Starting async job submit for node 69c01c93c3d73288b0d07b86 2026-03-24 16:45:25,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:25,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c93c3d73288b0d07b86 HTTP/1.1" 200 None 2026-03-24 16:45:25,513 - lava_callback - DEBUG - Processing node 69c01c93c3d73288b0d07b86: baseline-arm-baylibre 2026-03-24 16:45:25,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c93c3d73288b0d07b86 2026-03-24 16:45:25,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:25,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c93c3d73288b0d07b86 HTTP/1.1" 200 None 2026-03-24 16:45:25,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:25,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:25,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:25,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c93c3d73288b0d07b86 HTTP/1.1" 200 None 2026-03-24 16:45:25,621 - lava_callback - ERROR - Error processing callback for node 69c01c93c3d73288b0d07b86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:26,158 - lava_callback - INFO - Received callback for node 69c01c90c3d73288b0d07b6a INFO: 172.21.0.1:43840 - "POST /node/69c01c90c3d73288b0d07b6a HTTP/1.0" 202 Accepted 2026-03-24 16:45:26,167 - lava_callback - INFO - Starting async job submit for node 69c01c90c3d73288b0d07b6a 2026-03-24 16:45:26,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c90c3d73288b0d07b6a HTTP/1.1" 200 None 2026-03-24 16:45:26,200 - lava_callback - DEBUG - Processing node 69c01c90c3d73288b0d07b6a: baseline-arm-baylibre 2026-03-24 16:45:26,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c90c3d73288b0d07b6a 2026-03-24 16:45:26,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c90c3d73288b0d07b6a HTTP/1.1" 200 None 2026-03-24 16:45:26,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:26,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c90c3d73288b0d07b6a HTTP/1.1" 200 None 2026-03-24 16:45:26,299 - lava_callback - ERROR - Error processing callback for node 69c01c90c3d73288b0d07b6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43850 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:45:26,825 - lava_callback - INFO - Received callback for node 69c01c95c3d73288b0d07b8b INFO: 172.21.0.1:43854 - "POST /node/69c01c95c3d73288b0d07b8b HTTP/1.0" 202 Accepted 2026-03-24 16:45:26,834 - lava_callback - INFO - Starting async job submit for node 69c01c95c3d73288b0d07b8b 2026-03-24 16:45:26,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c95c3d73288b0d07b8b HTTP/1.1" 200 None 2026-03-24 16:45:26,863 - lava_callback - DEBUG - Processing node 69c01c95c3d73288b0d07b8b: baseline-arm-baylibre 2026-03-24 16:45:26,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c95c3d73288b0d07b8b 2026-03-24 16:45:26,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c95c3d73288b0d07b8b HTTP/1.1" 200 None 2026-03-24 16:45:26,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:26,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:26,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c95c3d73288b0d07b8b HTTP/1.1" 200 None 2026-03-24 16:45:26,971 - lava_callback - ERROR - Error processing callback for node 69c01c95c3d73288b0d07b8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:33,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:33,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 16:45:35,845 - lava_callback - INFO - Received callback for node 69c01c9ac3d73288b0d07b96 INFO: 172.21.0.1:47304 - "POST /node/69c01c9ac3d73288b0d07b96 HTTP/1.0" 202 Accepted 2026-03-24 16:45:35,854 - lava_callback - INFO - Starting async job submit for node 69c01c9ac3d73288b0d07b96 2026-03-24 16:45:35,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:35,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9ac3d73288b0d07b96 HTTP/1.1" 200 None 2026-03-24 16:45:35,888 - lava_callback - DEBUG - Processing node 69c01c9ac3d73288b0d07b96: baseline-arm-clabbe 2026-03-24 16:45:35,888 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c9ac3d73288b0d07b96 2026-03-24 16:45:35,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:35,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9ac3d73288b0d07b96 HTTP/1.1" 200 None 2026-03-24 16:45:35,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:35,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:45:35,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:35,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c9ac3d73288b0d07b96 HTTP/1.1" 200 None 2026-03-24 16:45:35,997 - lava_callback - ERROR - Error processing callback for node 69c01c9ac3d73288b0d07b96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:45:37,594 - lava_callback - INFO - Received callback for node 69c01c9dc3d73288b0d07ba3 INFO: 172.21.0.1:47326 - "POST /node/69c01c9dc3d73288b0d07ba3 HTTP/1.0" 202 Accepted 2026-03-24 16:45:37,602 - lava_callback - INFO - Starting async job submit for node 69c01c9dc3d73288b0d07ba3 2026-03-24 16:45:37,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:37,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9dc3d73288b0d07ba3 HTTP/1.1" 200 None 2026-03-24 16:45:37,635 - lava_callback - DEBUG - Processing node 69c01c9dc3d73288b0d07ba3: baseline-arm-clabbe 2026-03-24 16:45:37,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01c9dc3d73288b0d07ba3 2026-03-24 16:45:37,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:37,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9dc3d73288b0d07ba3 HTTP/1.1" 200 None 2026-03-24 16:45:37,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:37,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:37,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:37,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c9dc3d73288b0d07ba3 HTTP/1.1" 200 None 2026-03-24 16:45:37,742 - lava_callback - ERROR - Error processing callback for node 69c01c9dc3d73288b0d07ba3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:39,152 - lava_callback - INFO - Received callback for node 69c01ca0c3d73288b0d07bb0 INFO: 172.21.0.1:47342 - "POST /node/69c01ca0c3d73288b0d07bb0 HTTP/1.0" 202 Accepted 2026-03-24 16:45:39,163 - lava_callback - INFO - Starting async job submit for node 69c01ca0c3d73288b0d07bb0 2026-03-24 16:45:39,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:39,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca0c3d73288b0d07bb0 HTTP/1.1" 200 None 2026-03-24 16:45:39,192 - lava_callback - DEBUG - Processing node 69c01ca0c3d73288b0d07bb0: baseline-arm-clabbe 2026-03-24 16:45:39,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01ca0c3d73288b0d07bb0 2026-03-24 16:45:39,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:39,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca0c3d73288b0d07bb0 HTTP/1.1" 200 None 2026-03-24 16:45:39,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:39,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:39,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:39,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca0c3d73288b0d07bb0 HTTP/1.1" 200 None 2026-03-24 16:45:39,279 - lava_callback - ERROR - Error processing callback for node 69c01ca0c3d73288b0d07bb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:40,779 - lava_callback - INFO - Received callback for node 69c01ca1c3d73288b0d07bb3 INFO: 172.21.0.1:44312 - "POST /node/69c01ca1c3d73288b0d07bb3 HTTP/1.0" 202 Accepted 2026-03-24 16:45:40,790 - lava_callback - INFO - Starting async job submit for node 69c01ca1c3d73288b0d07bb3 2026-03-24 16:45:40,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:40,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca1c3d73288b0d07bb3 HTTP/1.1" 200 None 2026-03-24 16:45:40,835 - lava_callback - DEBUG - Processing node 69c01ca1c3d73288b0d07bb3: baseline-arm-clabbe 2026-03-24 16:45:40,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01ca1c3d73288b0d07bb3 2026-03-24 16:45:40,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:40,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca1c3d73288b0d07bb3 HTTP/1.1" 200 None 2026-03-24 16:45:40,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:40,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:40,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:40,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca1c3d73288b0d07bb3 HTTP/1.1" 200 None 2026-03-24 16:45:40,935 - lava_callback - ERROR - Error processing callback for node 69c01ca1c3d73288b0d07bb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:41,473 - lava_callback - INFO - Received callback for node 69c01ca2c3d73288b0d07bb6 INFO: 172.21.0.1:44318 - "POST /node/69c01ca2c3d73288b0d07bb6 HTTP/1.0" 202 Accepted 2026-03-24 16:45:41,482 - lava_callback - INFO - Starting async job submit for node 69c01ca2c3d73288b0d07bb6 2026-03-24 16:45:41,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:41,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca2c3d73288b0d07bb6 HTTP/1.1" 200 None 2026-03-24 16:45:41,515 - lava_callback - DEBUG - Processing node 69c01ca2c3d73288b0d07bb6: baseline-arm-clabbe 2026-03-24 16:45:41,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01ca2c3d73288b0d07bb6 2026-03-24 16:45:41,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:41,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca2c3d73288b0d07bb6 HTTP/1.1" 200 None 2026-03-24 16:45:41,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:41,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:41,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:41,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca2c3d73288b0d07bb6 HTTP/1.1" 200 None 2026-03-24 16:45:41,606 - lava_callback - ERROR - Error processing callback for node 69c01ca2c3d73288b0d07bb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 16:45:42,186 - lava_callback - INFO - Received callback for node 69c01ca5c3d73288b0d07bbf INFO: 172.21.0.1:44326 - "POST /node/69c01ca5c3d73288b0d07bbf HTTP/1.0" 202 Accepted 2026-03-24 16:45:42,195 - lava_callback - INFO - Starting async job submit for node 69c01ca5c3d73288b0d07bbf 2026-03-24 16:45:42,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:42,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca5c3d73288b0d07bbf HTTP/1.1" 200 None 2026-03-24 16:45:42,220 - lava_callback - DEBUG - Processing node 69c01ca5c3d73288b0d07bbf: baseline-arm-clabbe 2026-03-24 16:45:42,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01ca5c3d73288b0d07bbf 2026-03-24 16:45:42,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:42,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca5c3d73288b0d07bbf HTTP/1.1" 200 None 2026-03-24 16:45:42,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:42,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 16:45:42,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:42,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca5c3d73288b0d07bbf HTTP/1.1" 200 None 2026-03-24 16:45:42,341 - lava_callback - ERROR - Error processing callback for node 69c01ca5c3d73288b0d07bbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44338 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:45:55,849 - lava_callback - INFO - Received callback for node 69c01cc0c3d73288b0d07c0f INFO: 172.21.0.1:59664 - "POST /node/69c01cc0c3d73288b0d07c0f HTTP/1.0" 202 Accepted 2026-03-24 16:45:55,857 - lava_callback - INFO - Starting async job submit for node 69c01cc0c3d73288b0d07c0f 2026-03-24 16:45:55,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:55,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc0c3d73288b0d07c0f HTTP/1.1" 200 None 2026-03-24 16:45:55,885 - lava_callback - DEBUG - Processing node 69c01cc0c3d73288b0d07c0f: baseline-arm-clabbe 2026-03-24 16:45:55,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01cc0c3d73288b0d07c0f 2026-03-24 16:45:55,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:55,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc0c3d73288b0d07c0f HTTP/1.1" 200 None 2026-03-24 16:45:55,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:55,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:45:55,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:45:55,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cc0c3d73288b0d07c0f HTTP/1.1" 200 None 2026-03-24 16:45:55,986 - lava_callback - ERROR - Error processing callback for node 69c01cc0c3d73288b0d07c0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35018 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:46:14,546 - lava_callback - INFO - Received callback for node 69c2bfb76778148209fd35b8 INFO: 172.21.0.1:57108 - "POST /node/69c2bfb76778148209fd35b8 HTTP/1.0" 202 Accepted 2026-03-24 16:46:14,552 - lava_callback - INFO - Starting async job submit for node 69c2bfb76778148209fd35b8 2026-03-24 16:46:14,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:14,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb76778148209fd35b8 HTTP/1.1" 200 None 2026-03-24 16:46:14,598 - lava_callback - DEBUG - Processing node 69c2bfb76778148209fd35b8: baseline-x86 2026-03-24 16:46:14,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:14,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:14,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb76778148209fd35b8/log.txt.gz 2026-03-24 16:46:15,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:57114 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:46:15,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:15,239 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb76778148209fd35b8/lava-logs.yaml 2026-03-24 16:46:15,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:15,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:15,418 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb76778148209fd35b8/lava_callback.json.gz 2026-03-24 16:46:15,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:15,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb76778148209fd35b8 HTTP/1.1" 200 None 2026-03-24 16:46:15,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:15,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b97f6778148209fd3394 HTTP/1.1" 200 None 2026-03-24 16:46:15,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:15,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bfb76778148209fd35b8 HTTP/1.1" 200 None 2026-03-24 16:46:15,509 - lava_callback - INFO - Completed processing callback for node 69c2bfb76778148209fd35b8 INFO: 172.21.0.1:57118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45276 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:46:33,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:34,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 16:46:34,385 - lava_callback - INFO - Received callback for node 69c2bfb96778148209fd35c5 INFO: 172.21.0.1:49424 - "POST /node/69c2bfb96778148209fd35c5 HTTP/1.0" 202 Accepted 2026-03-24 16:46:34,391 - lava_callback - INFO - Starting async job submit for node 69c2bfb96778148209fd35c5 2026-03-24 16:46:34,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:34,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb96778148209fd35c5 HTTP/1.1" 200 None 2026-03-24 16:46:34,446 - lava_callback - DEBUG - Processing node 69c2bfb96778148209fd35c5: baseline-x86 2026-03-24 16:46:34,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:34,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:34,762 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb96778148209fd35c5/log.txt.gz 2026-03-24 16:46:34,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:35,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:35,077 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb96778148209fd35c5/lava-logs.yaml 2026-03-24 16:46:35,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:35,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:35,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb96778148209fd35c5/lava_callback.json.gz 2026-03-24 16:46:35,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:35,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb96778148209fd35c5 HTTP/1.1" 200 None 2026-03-24 16:46:35,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:35,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b97f6778148209fd3394 HTTP/1.1" 200 None 2026-03-24 16:46:35,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:35,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bfb96778148209fd35c5 HTTP/1.1" 200 None 2026-03-24 16:46:35,326 - lava_callback - INFO - Completed processing callback for node 69c2bfb96778148209fd35c5 2026-03-24 16:46:35,898 - lava_callback - INFO - Received callback for node 69c01ce6c3d73288b0d07c68 INFO: 172.21.0.1:49430 - "POST /node/69c01ce6c3d73288b0d07c68 HTTP/1.0" 202 Accepted 2026-03-24 16:46:35,904 - lava_callback - INFO - Starting async job submit for node 69c01ce6c3d73288b0d07c68 2026-03-24 16:46:35,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:35,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ce6c3d73288b0d07c68 HTTP/1.1" 200 None 2026-03-24 16:46:35,921 - lava_callback - DEBUG - Processing node 69c01ce6c3d73288b0d07c68: baseline-arm-clabbe 2026-03-24 16:46:35,921 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01ce6c3d73288b0d07c68 2026-03-24 16:46:35,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:35,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ce6c3d73288b0d07c68 HTTP/1.1" 200 None 2026-03-24 16:46:35,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:35,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 16:46:35,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:35,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ce6c3d73288b0d07c68 HTTP/1.1" 200 None 2026-03-24 16:46:35,971 - lava_callback - ERROR - Error processing callback for node 69c01ce6c3d73288b0d07c68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49438 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:46:45,938 - lava_callback - INFO - Received callback for node 69c2beca6778148209fd348b INFO: 172.21.0.1:43372 - "POST /node/69c2beca6778148209fd348b HTTP/1.0" 202 Accepted 2026-03-24 16:46:45,944 - lava_callback - INFO - Starting async job submit for node 69c2beca6778148209fd348b 2026-03-24 16:46:45,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:45,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beca6778148209fd348b HTTP/1.1" 200 None 2026-03-24 16:46:45,979 - lava_callback - DEBUG - Processing node 69c2beca6778148209fd348b: baseline-arm-clabbe 2026-03-24 16:46:46,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:46,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:46,303 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2beca6778148209fd348b/log.txt.gz 2026-03-24 16:46:46,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:43374 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:46:46,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:46,673 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2beca6778148209fd348b/lava-logs.yaml 2026-03-24 16:46:46,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:46,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:46,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2beca6778148209fd348b/lava_callback.json.gz 2026-03-24 16:46:46,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:46,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beca6778148209fd348b HTTP/1.1" 200 None 2026-03-24 16:46:46,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:46,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:46:46,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:46,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beca6778148209fd348b HTTP/1.1" 200 None 2026-03-24 16:46:46,910 - lava_callback - INFO - Completed processing callback for node 69c2beca6778148209fd348b INFO: 172.21.0.1:33704 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:46:58,386 - lava_callback - INFO - Received callback for node 69c2befc6778148209fd34fc INFO: 172.21.0.1:33718 - "POST /node/69c2befc6778148209fd34fc HTTP/1.0" 202 Accepted 2026-03-24 16:46:58,393 - lava_callback - INFO - Starting async job submit for node 69c2befc6778148209fd34fc 2026-03-24 16:46:58,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:46:58,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2befc6778148209fd34fc HTTP/1.1" 200 None 2026-03-24 16:46:58,443 - lava_callback - DEBUG - Processing node 69c2befc6778148209fd34fc: ltp-syscalls 2026-03-24 16:46:59,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:46:59,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:46:59,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c2befc6778148209fd34fc/log.txt.gz 2026-03-24 16:46:59,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:00,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:00,052 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c2befc6778148209fd34fc/lava-logs.yaml 2026-03-24 16:47:00,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:00,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:00,312 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c2befc6778148209fd34fc/lava_callback.json.gz 2026-03-24 16:47:00,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:00,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2befc6778148209fd34fc HTTP/1.1" 200 None 2026-03-24 16:47:00,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:00,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:47:00,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:00,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2befc6778148209fd34fc HTTP/1.1" 200 None 2026-03-24 16:47:00,414 - lava_callback - INFO - Completed processing callback for node 69c2befc6778148209fd34fc 2026-03-24 16:47:04,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:04,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 16:47:06,260 - lava_callback - INFO - Received callback for node 69c2bfb76778148209fd35bb INFO: 172.21.0.1:39938 - "POST /node/69c2bfb76778148209fd35bb HTTP/1.0" 202 Accepted 2026-03-24 16:47:06,267 - lava_callback - INFO - Starting async job submit for node 69c2bfb76778148209fd35bb 2026-03-24 16:47:06,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:06,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb76778148209fd35bb HTTP/1.1" 200 None 2026-03-24 16:47:06,303 - lava_callback - DEBUG - Processing node 69c2bfb76778148209fd35bb: baseline-x86 INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:47:06,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:06,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:06,868 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb76778148209fd35bb/log.txt.gz 2026-03-24 16:47:07,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:07,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:07,230 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb76778148209fd35bb/lava-logs.yaml 2026-03-24 16:47:07,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:07,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:07,368 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb76778148209fd35bb/lava_callback.json.gz 2026-03-24 16:47:07,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:07,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb76778148209fd35bb HTTP/1.1" 200 None 2026-03-24 16:47:07,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:07,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b97f6778148209fd3394 HTTP/1.1" 200 None 2026-03-24 16:47:07,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:07,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bfb76778148209fd35bb HTTP/1.1" 200 None 2026-03-24 16:47:07,455 - lava_callback - INFO - Completed processing callback for node 69c2bfb76778148209fd35bb 2026-03-24 16:47:09,348 - lava_callback - INFO - Received callback for node 69c2bfb86778148209fd35c1 INFO: 172.21.0.1:39960 - "POST /node/69c2bfb86778148209fd35c1 HTTP/1.0" 202 Accepted 2026-03-24 16:47:09,354 - lava_callback - INFO - Starting async job submit for node 69c2bfb86778148209fd35c1 2026-03-24 16:47:09,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:09,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb86778148209fd35c1 HTTP/1.1" 200 None 2026-03-24 16:47:09,388 - lava_callback - DEBUG - Processing node 69c2bfb86778148209fd35c1: baseline-x86 2026-03-24 16:47:09,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:09,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:09,763 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb86778148209fd35c1/log.txt.gz 2026-03-24 16:47:09,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:10,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:10,096 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb86778148209fd35c1/lava-logs.yaml 2026-03-24 16:47:10,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:47:10,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:47:10,241 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfb86778148209fd35c1/lava_callback.json.gz 2026-03-24 16:47:10,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:10,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfb86778148209fd35c1 HTTP/1.1" 200 None 2026-03-24 16:47:10,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:10,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b97f6778148209fd3394 HTTP/1.1" 200 None 2026-03-24 16:47:10,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:10,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bfb86778148209fd35c1 HTTP/1.1" 200 None 2026-03-24 16:47:10,330 - lava_callback - INFO - Completed processing callback for node 69c2bfb86778148209fd35c1 INFO: 172.21.0.1:54596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45462 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:47:25,010 - lava_callback - INFO - Received callback for node 69c01d08c3d73288b0d07c96 INFO: 172.21.0.1:45468 - "POST /node/69c01d08c3d73288b0d07c96 HTTP/1.0" 202 Accepted 2026-03-24 16:47:25,016 - lava_callback - INFO - Starting async job submit for node 69c01d08c3d73288b0d07c96 2026-03-24 16:47:25,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:25,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01d08c3d73288b0d07c96 HTTP/1.1" 200 None 2026-03-24 16:47:25,032 - lava_callback - DEBUG - Processing node 69c01d08c3d73288b0d07c96: baseline-arm-baylibre 2026-03-24 16:47:25,032 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01d08c3d73288b0d07c96 2026-03-24 16:47:25,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:25,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01d08c3d73288b0d07c96 HTTP/1.1" 200 None 2026-03-24 16:47:25,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:25,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:47:25,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:25,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01d08c3d73288b0d07c96 HTTP/1.1" 200 None 2026-03-24 16:47:25,111 - lava_callback - ERROR - Error processing callback for node 69c01d08c3d73288b0d07c96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45482 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:47:34,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:34,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41220 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:47:45,007 - lava_callback - INFO - Received callback for node 69c01d2ec3d73288b0d07cb8 INFO: 172.21.0.1:39860 - "POST /node/69c01d2ec3d73288b0d07cb8 HTTP/1.0" 202 Accepted 2026-03-24 16:47:45,013 - lava_callback - INFO - Starting async job submit for node 69c01d2ec3d73288b0d07cb8 2026-03-24 16:47:45,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:45,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01d2ec3d73288b0d07cb8 HTTP/1.1" 200 None 2026-03-24 16:47:45,029 - lava_callback - DEBUG - Processing node 69c01d2ec3d73288b0d07cb8: baseline-arm-baylibre 2026-03-24 16:47:45,029 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01d2ec3d73288b0d07cb8 2026-03-24 16:47:45,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:45,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01d2ec3d73288b0d07cb8 HTTP/1.1" 200 None 2026-03-24 16:47:45,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:45,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:47:45,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:47:45,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01d2ec3d73288b0d07cb8 HTTP/1.1" 200 None 2026-03-24 16:47:45,086 - lava_callback - ERROR - Error processing callback for node 69c01d2ec3d73288b0d07cb8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49132 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:48:25,062 - lava_callback - INFO - Received callback for node 69c2becb6778148209fd348e INFO: 172.21.0.1:49138 - "POST /node/69c2becb6778148209fd348e HTTP/1.0" 202 Accepted 2026-03-24 16:48:25,068 - lava_callback - INFO - Starting async job submit for node 69c2becb6778148209fd348e 2026-03-24 16:48:25,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:48:25,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2becb6778148209fd348e HTTP/1.1" 200 None 2026-03-24 16:48:25,105 - lava_callback - DEBUG - Processing node 69c2becb6778148209fd348e: baseline-arm-clabbe 2026-03-24 16:48:25,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:48:25,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:48:25,476 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2becb6778148209fd348e/log.txt.gz 2026-03-24 16:48:25,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:48:25,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:48:25,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2becb6778148209fd348e/lava-logs.yaml 2026-03-24 16:48:25,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:48:25,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:48:25,962 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c2becb6778148209fd348e/lava_callback.json.gz 2026-03-24 16:48:25,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:48:25,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2becb6778148209fd348e HTTP/1.1" 200 None 2026-03-24 16:48:26,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:48:26,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:48:26,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:48:26,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2becb6778148209fd348e HTTP/1.1" 200 None 2026-03-24 16:48:26,082 - lava_callback - INFO - Completed processing callback for node 69c2becb6778148209fd348e INFO: 172.21.0.1:49140 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:48:34,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:48:34,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48054 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:49:04,392 - lava_callback - INFO - Received callback for node 69c2c0736778148209fd3640 INFO: 172.21.0.1:60022 - "POST /node/69c2c0736778148209fd3640 HTTP/1.0" 202 Accepted 2026-03-24 16:49:04,398 - lava_callback - INFO - Starting async job submit for node 69c2c0736778148209fd3640 2026-03-24 16:49:04,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:49:04,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0736778148209fd3640 HTTP/1.1" 200 None 2026-03-24 16:49:04,417 - lava_callback - DEBUG - Processing node 69c2c0736778148209fd3640: baseline-arm 2026-03-24 16:49:04,422 - lava_callback - WARNING - Failed to upload log 2026-03-24 16:49:04,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:49:04,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:49:04,608 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2c0736778148209fd3640/lava-logs.yaml 2026-03-24 16:49:04,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:49:04,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:49:04,745 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2c0736778148209fd3640/lava_callback.json.gz 2026-03-24 16:49:04,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:49:04,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0736778148209fd3640 HTTP/1.1" 200 None 2026-03-24 16:49:04,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:49:04,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:49:04,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:49:04,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c0736778148209fd3640 HTTP/1.1" 200 None 2026-03-24 16:49:04,821 - lava_callback - INFO - Completed processing callback for node 69c2c0736778148209fd3640 INFO: 172.21.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54712 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:49:34,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:49:34,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57692 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:50:25,179 - lava_callback - INFO - Received callback for node 69c01dc6c3d73288b0d07d0d INFO: 172.21.0.1:57704 - "POST /node/69c01dc6c3d73288b0d07d0d HTTP/1.0" 202 Accepted 2026-03-24 16:50:25,185 - lava_callback - INFO - Starting async job submit for node 69c01dc6c3d73288b0d07d0d 2026-03-24 16:50:25,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:25,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01dc6c3d73288b0d07d0d HTTP/1.1" 200 None 2026-03-24 16:50:25,201 - lava_callback - DEBUG - Processing node 69c01dc6c3d73288b0d07d0d: baseline-arm-baylibre 2026-03-24 16:50:25,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01dc6c3d73288b0d07d0d 2026-03-24 16:50:25,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:25,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01dc6c3d73288b0d07d0d HTTP/1.1" 200 None 2026-03-24 16:50:25,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:25,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:50:25,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:25,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01dc6c3d73288b0d07d0d HTTP/1.1" 200 None 2026-03-24 16:50:25,278 - lava_callback - ERROR - Error processing callback for node 69c01dc6c3d73288b0d07d0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57710 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:50:28,236 - lava_callback - INFO - Received callback for node 69c2bc336778148209fd33b7 INFO: 172.21.0.1:57720 - "POST /node/69c2bc336778148209fd33b7 HTTP/1.0" 202 Accepted 2026-03-24 16:50:28,244 - lava_callback - INFO - Starting async job submit for node 69c2bc336778148209fd33b7 2026-03-24 16:50:28,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:28,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bc336778148209fd33b7 HTTP/1.1" 200 None 2026-03-24 16:50:28,335 - lava_callback - DEBUG - Processing node 69c2bc336778148209fd33b7: ltp-smoke 2026-03-24 16:50:28,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:50:28,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:50:28,954 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2bc336778148209fd33b7/log.txt.gz 2026-03-24 16:50:29,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:50:29,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:50:29,469 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2bc336778148209fd33b7/lava-logs.yaml 2026-03-24 16:50:29,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:50:29,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:50:29,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2bc336778148209fd33b7/lava_callback.json.gz 2026-03-24 16:50:29,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:29,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bc336778148209fd33b7 HTTP/1.1" 200 None 2026-03-24 16:50:29,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:29,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9546778148209fd3372 HTTP/1.1" 200 None 2026-03-24 16:50:29,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:29,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bc336778148209fd33b7 HTTP/1.1" 200 None 2026-03-24 16:50:29,944 - lava_callback - INFO - Completed processing callback for node 69c2bc336778148209fd33b7 2026-03-24 16:50:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:50:34,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56584 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:51:36,296 - lava_callback - INFO - Received callback for node 69c2c0cd6778148209fd365d INFO: 172.21.0.1:51766 - "POST /node/69c2c0cd6778148209fd365d HTTP/1.0" 202 Accepted 2026-03-24 16:51:36,302 - lava_callback - INFO - Starting async job submit for node 69c2c0cd6778148209fd365d 2026-03-24 16:51:36,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:51:36,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0cd6778148209fd365d HTTP/1.1" 200 None 2026-03-24 16:51:36,347 - lava_callback - DEBUG - Processing node 69c2c0cd6778148209fd365d: baseline-arm-pengutronix INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:51:36,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:51:36,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:51:36,691 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2c0cd6778148209fd365d/log.txt.gz 2026-03-24 16:51:36,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:51:37,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:51:37,053 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2c0cd6778148209fd365d/lava-logs.yaml 2026-03-24 16:51:37,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:51:37,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:51:37,221 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2c0cd6778148209fd365d/lava_callback.json.gz 2026-03-24 16:51:37,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:51:37,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c0cd6778148209fd365d HTTP/1.1" 200 None 2026-03-24 16:51:37,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:51:37,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:51:37,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:51:37,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c0cd6778148209fd365d HTTP/1.1" 200 None 2026-03-24 16:51:37,321 - lava_callback - INFO - Completed processing callback for node 69c2c0cd6778148209fd365d INFO: 172.21.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35738 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:52:04,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:52:04,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41378 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:53:25,623 - lava_callback - INFO - Received callback for node 69c01e81c3d73288b0d07d81 INFO: 172.21.0.1:41392 - "POST /node/69c01e81c3d73288b0d07d81 HTTP/1.0" 202 Accepted 2026-03-24 16:53:25,631 - lava_callback - INFO - Starting async job submit for node 69c01e81c3d73288b0d07d81 2026-03-24 16:53:25,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:53:25,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01e81c3d73288b0d07d81 HTTP/1.1" 200 None 2026-03-24 16:53:25,657 - lava_callback - DEBUG - Processing node 69c01e81c3d73288b0d07d81: baseline-arm-baylibre 2026-03-24 16:53:25,657 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01e81c3d73288b0d07d81 2026-03-24 16:53:25,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:53:25,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01e81c3d73288b0d07d81 HTTP/1.1" 200 None 2026-03-24 16:53:25,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:53:25,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:53:25,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:53:25,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01e81c3d73288b0d07d81 HTTP/1.1" 200 None 2026-03-24 16:53:25,721 - lava_callback - ERROR - Error processing callback for node 69c01e81c3d73288b0d07d81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57690 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:54:16,467 - lava_callback - INFO - Received callback for node 69c01ea8c3d73288b0d07d90 INFO: 172.21.0.1:57692 - "POST /node/69c01ea8c3d73288b0d07d90 HTTP/1.0" 202 Accepted 2026-03-24 16:54:16,473 - lava_callback - INFO - Starting async job submit for node 69c01ea8c3d73288b0d07d90 2026-03-24 16:54:16,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:57696 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:54:16,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ea8c3d73288b0d07d90 HTTP/1.1" 200 None 2026-03-24 16:54:16,490 - lava_callback - DEBUG - Processing node 69c01ea8c3d73288b0d07d90: baseline-arm-clabbe 2026-03-24 16:54:16,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01ea8c3d73288b0d07d90 2026-03-24 16:54:16,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:54:16,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ea8c3d73288b0d07d90 HTTP/1.1" 200 None 2026-03-24 16:54:16,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:54:16,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:54:16,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:54:16,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ea8c3d73288b0d07d90 HTTP/1.1" 200 None 2026-03-24 16:54:16,546 - lava_callback - ERROR - Error processing callback for node 69c01ea8c3d73288b0d07d90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44522 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:55:05,868 - lava_callback - INFO - Received callback for node 69c01ed6c3d73288b0d07dac INFO: 172.21.0.1:40654 - "POST /node/69c01ed6c3d73288b0d07dac HTTP/1.0" 202 Accepted 2026-03-24 16:55:05,873 - lava_callback - INFO - Starting async job submit for node 69c01ed6c3d73288b0d07dac 2026-03-24 16:55:05,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:55:05,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ed6c3d73288b0d07dac HTTP/1.1" 200 None 2026-03-24 16:55:05,894 - lava_callback - DEBUG - Processing node 69c01ed6c3d73288b0d07dac: baseline-arm64-baylibre 2026-03-24 16:55:05,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01ed6c3d73288b0d07dac 2026-03-24 16:55:05,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:55:05,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ed6c3d73288b0d07dac HTTP/1.1" 200 None 2026-03-24 16:55:05,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:55:05,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:55:05,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:55:05,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ed6c3d73288b0d07dac HTTP/1.1" 200 None 2026-03-24 16:55:05,965 - lava_callback - ERROR - Error processing callback for node 69c01ed6c3d73288b0d07dac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33562 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 16:56:16,482 - lava_callback - INFO - Received callback for node 69c01f19c3d73288b0d07dcd INFO: 172.21.0.1:33578 - "POST /node/69c01f19c3d73288b0d07dcd HTTP/1.0" 202 Accepted 2026-03-24 16:56:16,488 - lava_callback - INFO - Starting async job submit for node 69c01f19c3d73288b0d07dcd 2026-03-24 16:56:16,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:56:16,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f19c3d73288b0d07dcd HTTP/1.1" 200 None 2026-03-24 16:56:16,504 - lava_callback - DEBUG - Processing node 69c01f19c3d73288b0d07dcd: baseline-arm-clabbe 2026-03-24 16:56:16,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01f19c3d73288b0d07dcd 2026-03-24 16:56:16,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:33580 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:56:16,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f19c3d73288b0d07dcd HTTP/1.1" 200 None 2026-03-24 16:56:16,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:56:16,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:56:16,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:56:16,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01f19c3d73288b0d07dcd HTTP/1.1" 200 None 2026-03-24 16:56:16,568 - lava_callback - ERROR - Error processing callback for node 69c01f19c3d73288b0d07dcd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:56:36,420 - lava_callback - INFO - Received callback for node 69c01f3fc3d73288b0d07dd7 INFO: 172.21.0.1:59960 - "POST /node/69c01f3fc3d73288b0d07dd7 HTTP/1.0" 202 Accepted 2026-03-24 16:56:36,425 - lava_callback - INFO - Starting async job submit for node 69c01f3fc3d73288b0d07dd7 2026-03-24 16:56:36,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:56:36,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f3fc3d73288b0d07dd7 HTTP/1.1" 200 None 2026-03-24 16:56:36,443 - lava_callback - DEBUG - Processing node 69c01f3fc3d73288b0d07dd7: baseline-arm-clabbe 2026-03-24 16:56:36,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01f3fc3d73288b0d07dd7 2026-03-24 16:56:36,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:56:36,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f3fc3d73288b0d07dd7 HTTP/1.1" 200 None 2026-03-24 16:56:36,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:56:36,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:56:36,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:59976 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:56:36,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01f3fc3d73288b0d07dd7 HTTP/1.1" 200 None 2026-03-24 16:56:36,516 - lava_callback - ERROR - Error processing callback for node 69c01f3fc3d73288b0d07dd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35348 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:57:16,487 - lava_callback - INFO - Received callback for node 69c01f64c3d73288b0d07de2 INFO: 172.21.0.1:35350 - "POST /node/69c01f64c3d73288b0d07de2 HTTP/1.0" 202 Accepted 2026-03-24 16:57:16,496 - lava_callback - INFO - Starting async job submit for node 69c01f64c3d73288b0d07de2 2026-03-24 16:57:16,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:16,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f64c3d73288b0d07de2 HTTP/1.1" 200 None 2026-03-24 16:57:16,534 - lava_callback - DEBUG - Processing node 69c01f64c3d73288b0d07de2: baseline-arm-clabbe 2026-03-24 16:57:16,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01f64c3d73288b0d07de2 2026-03-24 16:57:16,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:16,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f64c3d73288b0d07de2 HTTP/1.1" 200 None 2026-03-24 16:57:16,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:16,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:57:16,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:16,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01f64c3d73288b0d07de2 HTTP/1.1" 200 None 2026-03-24 16:57:16,656 - lava_callback - ERROR - Error processing callback for node 69c01f64c3d73288b0d07de2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60552 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:57:56,508 - lava_callback - INFO - Received callback for node 69c01f89c3d73288b0d07df4 INFO: 172.21.0.1:60556 - "POST /node/69c01f89c3d73288b0d07df4 HTTP/1.0" 202 Accepted 2026-03-24 16:57:56,518 - lava_callback - INFO - Starting async job submit for node 69c01f89c3d73288b0d07df4 2026-03-24 16:57:56,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:56,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f89c3d73288b0d07df4 HTTP/1.1" 200 None 2026-03-24 16:57:56,549 - lava_callback - DEBUG - Processing node 69c01f89c3d73288b0d07df4: baseline-arm-clabbe 2026-03-24 16:57:56,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c01f89c3d73288b0d07df4 2026-03-24 16:57:56,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:56,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01f89c3d73288b0d07df4 HTTP/1.1" 200 None 2026-03-24 16:57:56,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:56,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-24 16:57:56,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:57:56,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01f89c3d73288b0d07df4 HTTP/1.1" 200 None 2026-03-24 16:57:56,655 - lava_callback - ERROR - Error processing callback for node 69c01f89c3d73288b0d07df4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35434 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:58:54,638 - lava_callback - INFO - Received callback for node 69c2bead6778148209fd3412 INFO: 172.21.0.1:36140 - "POST /node/69c2bead6778148209fd3412 HTTP/1.0" 202 Accepted 2026-03-24 16:58:54,645 - lava_callback - INFO - Starting async job submit for node 69c2bead6778148209fd3412 2026-03-24 16:58:54,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:54,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bead6778148209fd3412 HTTP/1.1" 200 None 2026-03-24 16:58:54,757 - lava_callback - DEBUG - Processing node 69c2bead6778148209fd3412: baseline-arm 2026-03-24 16:58:55,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:58:55,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:58:55,577 - lava_callback - INFO - Received callback for node 69c2c2a96778148209fd5dc6 2026-03-24 16:58:55,587 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bead6778148209fd3412/log.txt.gz 2026-03-24 16:58:55,589 - lava_callback - INFO - Starting async job submit for node 69c2c2a96778148209fd5dc6 INFO: 172.21.0.1:36152 - "POST /node/69c2c2a96778148209fd5dc6 HTTP/1.0" 202 Accepted 2026-03-24 16:58:55,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:55,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2a96778148209fd5dc6 HTTP/1.1" 200 None 2026-03-24 16:58:55,753 - lava_callback - DEBUG - Processing node 69c2c2a96778148209fd5dc6: baseline-arm64 2026-03-24 16:58:56,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:58:56,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:58:56,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:58:56,391 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2a96778148209fd5dc6/log.txt.gz 2026-03-24 16:58:56,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:36164 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:58:56,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:58:56,611 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bead6778148209fd3412/lava-logs.yaml 2026-03-24 16:58:56,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:58:56,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:58:56,809 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2a96778148209fd5dc6/lava-logs.yaml 2026-03-24 16:58:56,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:58:56,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:58:56,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2bead6778148209fd3412/lava_callback.json.gz DEBUG: Setup failed for 69c2bead6778148209fd3412. Transitting job node result: incomplete -> fail 2026-03-24 16:58:56,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:56,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bead6778148209fd3412 HTTP/1.1" 200 None 2026-03-24 16:58:56,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:56,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 16:58:56,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:56,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:58:56,988 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2a96778148209fd5dc6/lava_callback.json.gz 2026-03-24 16:58:57,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bead6778148209fd3412 HTTP/1.1" 200 None 2026-03-24 16:58:57,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:57,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2a96778148209fd5dc6 HTTP/1.1" 200 None 2026-03-24 16:58:57,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:57,154 - lava_callback - INFO - Completed processing callback for node 69c2bead6778148209fd3412 2026-03-24 16:58:57,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 16:58:57,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:58:57,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2a96778148209fd5dc6 HTTP/1.1" 200 None 2026-03-24 16:58:57,231 - lava_callback - INFO - Completed processing callback for node 69c2c2a96778148209fd5dc6 2026-03-24 16:59:04,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:04,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56534 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:59:08,189 - lava_callback - INFO - Received callback for node 69c2c2c66778148209fd5e11 INFO: 172.21.0.1:56544 - "POST /node/69c2c2c66778148209fd5e11 HTTP/1.0" 202 Accepted 2026-03-24 16:59:08,195 - lava_callback - INFO - Starting async job submit for node 69c2c2c66778148209fd5e11 2026-03-24 16:59:08,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:08,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c66778148209fd5e11 HTTP/1.1" 200 None 2026-03-24 16:59:08,212 - lava_callback - DEBUG - Processing node 69c2c2c66778148209fd5e11: baseline-nfs-arm64 2026-03-24 16:59:08,214 - lava_callback - WARNING - Failed to upload log 2026-03-24 16:59:08,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:08,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:08,346 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c66778148209fd5e11/lava-logs.yaml 2026-03-24 16:59:08,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:08,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:08,463 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c66778148209fd5e11/lava_callback.json.gz 2026-03-24 16:59:08,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:08,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c66778148209fd5e11 HTTP/1.1" 200 None 2026-03-24 16:59:08,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:08,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 16:59:08,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:08,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c66778148209fd5e11 HTTP/1.1" 200 None 2026-03-24 16:59:08,524 - lava_callback - INFO - Completed processing callback for node 69c2c2c66778148209fd5e11 INFO: 172.21.0.1:47502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47504 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:59:16,621 - lava_callback - INFO - Received callback for node 69c2c2b56778148209fd5de7 INFO: 172.21.0.1:47520 - "POST /node/69c2c2b56778148209fd5de7 HTTP/1.0" 202 Accepted 2026-03-24 16:59:16,649 - lava_callback - INFO - Starting async job submit for node 69c2c2b56778148209fd5de7 2026-03-24 16:59:16,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:16,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b56778148209fd5de7 HTTP/1.1" 200 None 2026-03-24 16:59:16,805 - lava_callback - DEBUG - Processing node 69c2c2b56778148209fd5de7: baseline-arm64 2026-03-24 16:59:17,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:17,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:17,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b56778148209fd5de7/log.txt.gz 2026-03-24 16:59:17,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:18,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:18,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b56778148209fd5de7/lava-logs.yaml 2026-03-24 16:59:18,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:18,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:18,980 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b56778148209fd5de7/lava_callback.json.gz 2026-03-24 16:59:19,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:19,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b56778148209fd5de7 HTTP/1.1" 200 None 2026-03-24 16:59:19,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:19,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 16:59:19,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:19,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b56778148209fd5de7 HTTP/1.1" 200 None 2026-03-24 16:59:19,127 - lava_callback - INFO - Completed processing callback for node 69c2c2b56778148209fd5de7 INFO: 172.21.0.1:51318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51322 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:59:34,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:34,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 16:59:35,342 - lava_callback - INFO - Received callback for node 69c2c2ad6778148209fd5dd2 INFO: 172.21.0.1:47204 - "POST /node/69c2c2ad6778148209fd5dd2 HTTP/1.0" 202 Accepted 2026-03-24 16:59:35,352 - lava_callback - INFO - Starting async job submit for node 69c2c2ad6778148209fd5dd2 2026-03-24 16:59:35,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:35,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ad6778148209fd5dd2 HTTP/1.1" 200 None 2026-03-24 16:59:35,434 - lava_callback - DEBUG - Processing node 69c2c2ad6778148209fd5dd2: baseline-arm64 2026-03-24 16:59:35,673 - lava_callback - WARNING - Failed to upload log 2026-03-24 16:59:35,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:36,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:36,318 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ad6778148209fd5dd2/lava-logs.yaml 2026-03-24 16:59:36,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:47216 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:59:36,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:36,740 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ad6778148209fd5dd2/lava_callback.json.gz 2026-03-24 16:59:36,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:36,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ad6778148209fd5dd2 HTTP/1.1" 200 None 2026-03-24 16:59:36,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:36,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 16:59:36,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:36,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ad6778148209fd5dd2 HTTP/1.1" 200 None 2026-03-24 16:59:37,014 - lava_callback - INFO - Completed processing callback for node 69c2c2ad6778148209fd5dd2 2026-03-24 16:59:37,817 - lava_callback - INFO - Received callback for node 69c2c2ab6778148209fd5dcc INFO: 172.21.0.1:47218 - "POST /node/69c2c2ab6778148209fd5dcc HTTP/1.0" 202 Accepted 2026-03-24 16:59:37,830 - lava_callback - INFO - Starting async job submit for node 69c2c2ab6778148209fd5dcc 2026-03-24 16:59:37,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:37,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ab6778148209fd5dcc HTTP/1.1" 200 None 2026-03-24 16:59:37,906 - lava_callback - DEBUG - Processing node 69c2c2ab6778148209fd5dcc: baseline-arm64 2026-03-24 16:59:38,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:38,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:38,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ab6778148209fd5dcc/log.txt.gz 2026-03-24 16:59:39,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:39,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:39,882 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ab6778148209fd5dcc/lava-logs.yaml 2026-03-24 16:59:39,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:40,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:40,389 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ab6778148209fd5dcc/lava_callback.json.gz 2026-03-24 16:59:40,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:40,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ab6778148209fd5dcc HTTP/1.1" 200 None 2026-03-24 16:59:40,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:40,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 16:59:40,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:40,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ab6778148209fd5dcc HTTP/1.1" 200 None 2026-03-24 16:59:40,523 - lava_callback - INFO - Completed processing callback for node 69c2c2ab6778148209fd5dcc INFO: 172.21.0.1:39304 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:59:47,834 - lava_callback - INFO - Received callback for node 69c2c2ae6778148209fd5dd5 INFO: 172.21.0.1:39314 - "POST /node/69c2c2ae6778148209fd5dd5 HTTP/1.0" 202 Accepted 2026-03-24 16:59:47,849 - lava_callback - INFO - Starting async job submit for node 69c2c2ae6778148209fd5dd5 2026-03-24 16:59:47,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:47,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ae6778148209fd5dd5 HTTP/1.1" 200 None 2026-03-24 16:59:47,934 - lava_callback - DEBUG - Processing node 69c2c2ae6778148209fd5dd5: baseline-arm64 2026-03-24 16:59:48,306 - lava_callback - WARNING - Failed to upload log 2026-03-24 16:59:48,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:48,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:48,930 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ae6778148209fd5dd5/lava-logs.yaml 2026-03-24 16:59:48,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:49,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:49,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ae6778148209fd5dd5/lava_callback.json.gz 2026-03-24 16:59:49,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:49,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ae6778148209fd5dd5 HTTP/1.1" 200 None 2026-03-24 16:59:49,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:49,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 16:59:49,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:49,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ae6778148209fd5dd5 HTTP/1.1" 200 None 2026-03-24 16:59:49,660 - lava_callback - INFO - Completed processing callback for node 69c2c2ae6778148209fd5dd5 2026-03-24 16:59:55,679 - lava_callback - INFO - Received callback for node 69c2c2b36778148209fd5de1 INFO: 172.21.0.1:35578 - "POST /node/69c2c2b36778148209fd5de1 HTTP/1.0" 202 Accepted 2026-03-24 16:59:55,690 - lava_callback - INFO - Starting async job submit for node 69c2c2b36778148209fd5de1 2026-03-24 16:59:55,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:55,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b36778148209fd5de1 HTTP/1.1" 200 None 2026-03-24 16:59:55,768 - lava_callback - DEBUG - Processing node 69c2c2b36778148209fd5de1: baseline-arm64 2026-03-24 16:59:56,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35592 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:59:57,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:57,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b36778148209fd5de1/log.txt.gz 2026-03-24 16:59:57,371 - lava_callback - INFO - Received callback for node 69c2c2aa6778148209fd5dc9 INFO: 172.21.0.1:35604 - "POST /node/69c2c2aa6778148209fd5dc9 HTTP/1.0" 202 Accepted 2026-03-24 16:59:57,392 - lava_callback - INFO - Starting async job submit for node 69c2c2aa6778148209fd5dc9 2026-03-24 16:59:57,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:57,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2aa6778148209fd5dc9 HTTP/1.1" 200 None 2026-03-24 16:59:57,511 - lava_callback - DEBUG - Processing node 69c2c2aa6778148209fd5dc9: baseline-arm64 2026-03-24 16:59:57,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:57,782 - lava_callback - INFO - Received callback for node 69c2c2b26778148209fd5dde 2026-03-24 16:59:57,803 - lava_callback - INFO - Starting async job submit for node 69c2c2b26778148209fd5dde INFO: 172.21.0.1:35606 - "POST /node/69c2c2b26778148209fd5dde HTTP/1.0" 202 Accepted 2026-03-24 16:59:57,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:57,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b26778148209fd5dde HTTP/1.1" 200 None 2026-03-24 16:59:57,961 - lava_callback - DEBUG - Processing node 69c2c2b26778148209fd5dde: baseline-arm64 2026-03-24 16:59:58,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:58,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:58,371 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b36778148209fd5de1/lava-logs.yaml 2026-03-24 16:59:58,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:58,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:58,506 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2aa6778148209fd5dc9/log.txt.gz 2026-03-24 16:59:58,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:58,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:59,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:59,186 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b26778148209fd5dde/log.txt.gz 2026-03-24 16:59:59,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:59,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b36778148209fd5de1/lava_callback.json.gz 2026-03-24 16:59:59,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:59,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b36778148209fd5de1 HTTP/1.1" 200 None 2026-03-24 16:59:59,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:59,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 16:59:59,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:59:59,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b36778148209fd5de1 HTTP/1.1" 200 None 2026-03-24 16:59:59,587 - lava_callback - INFO - Completed processing callback for node 69c2c2b36778148209fd5de1 2026-03-24 16:59:59,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 16:59:59,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 16:59:59,908 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2aa6778148209fd5dc9/lava-logs.yaml 2026-03-24 16:59:59,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:00,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:00,518 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2aa6778148209fd5dc9/lava_callback.json.gz 2026-03-24 17:00:00,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:00,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2aa6778148209fd5dc9 HTTP/1.1" 200 None 2026-03-24 17:00:00,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:00,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:00,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:00,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:00,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b26778148209fd5dde/lava-logs.yaml 2026-03-24 17:00:00,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2aa6778148209fd5dc9 HTTP/1.1" 200 None 2026-03-24 17:00:00,691 - lava_callback - INFO - Completed processing callback for node 69c2c2aa6778148209fd5dc9 2026-03-24 17:00:00,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:01,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:01,208 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b26778148209fd5dde/lava_callback.json.gz 2026-03-24 17:00:01,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:01,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b26778148209fd5dde HTTP/1.1" 200 None 2026-03-24 17:00:01,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:01,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:01,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:01,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b26778148209fd5dde HTTP/1.1" 200 None 2026-03-24 17:00:01,336 - lava_callback - INFO - Completed processing callback for node 69c2c2b26778148209fd5dde 2026-03-24 17:00:04,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:04,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 17:00:04,915 - lava_callback - INFO - Received callback for node 69c2c2ac6778148209fd5dcf INFO: 172.21.0.1:42466 - "POST /node/69c2c2ac6778148209fd5dcf HTTP/1.0" 202 Accepted 2026-03-24 17:00:04,925 - lava_callback - INFO - Starting async job submit for node 69c2c2ac6778148209fd5dcf 2026-03-24 17:00:04,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:04,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ac6778148209fd5dcf HTTP/1.1" 200 None 2026-03-24 17:00:04,998 - lava_callback - DEBUG - Processing node 69c2c2ac6778148209fd5dcf: baseline-arm64 2026-03-24 17:00:05,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:05,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:05,863 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ac6778148209fd5dcf/log.txt.gz 2026-03-24 17:00:06,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:06,332 - lava_callback - INFO - Received callback for node 69c2c2c16778148209fd5e05 INFO: 172.21.0.1:42472 - "POST /node/69c2c2c16778148209fd5e05 HTTP/1.0" 202 Accepted 2026-03-24 17:00:06,342 - lava_callback - INFO - Starting async job submit for node 69c2c2c16778148209fd5e05 2026-03-24 17:00:06,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:06,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c16778148209fd5e05 HTTP/1.1" 200 None 2026-03-24 17:00:06,430 - lava_callback - DEBUG - Processing node 69c2c2c16778148209fd5e05: baseline-nfs-arm64 INFO: 172.21.0.1:42486 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:00:06,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:06,758 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ac6778148209fd5dcf/lava-logs.yaml 2026-03-24 17:00:06,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:07,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:07,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:07,203 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ac6778148209fd5dcf/lava_callback.json.gz 2026-03-24 17:00:07,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:07,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ac6778148209fd5dcf HTTP/1.1" 200 None 2026-03-24 17:00:07,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:07,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:07,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:07,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ac6778148209fd5dcf HTTP/1.1" 200 None 2026-03-24 17:00:07,473 - lava_callback - INFO - Completed processing callback for node 69c2c2ac6778148209fd5dcf 2026-03-24 17:00:07,532 - lava_callback - INFO - Received callback for node 69c2c2b46778148209fd5de4 INFO: 172.21.0.1:42502 - "POST /node/69c2c2b46778148209fd5de4 HTTP/1.0" 202 Accepted 2026-03-24 17:00:07,553 - lava_callback - INFO - Starting async job submit for node 69c2c2b46778148209fd5de4 2026-03-24 17:00:07,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:07,622 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c16778148209fd5e05/log.txt.gz 2026-03-24 17:00:07,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:07,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b46778148209fd5de4 HTTP/1.1" 200 None 2026-03-24 17:00:07,779 - lava_callback - DEBUG - Processing node 69c2c2b46778148209fd5de4: baseline-arm64 2026-03-24 17:00:08,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:08,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:09,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:09,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c16778148209fd5e05/lava-logs.yaml 2026-03-24 17:00:09,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:09,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:09,356 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b46778148209fd5de4/log.txt.gz 2026-03-24 17:00:09,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:09,455 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c16778148209fd5e05/lava_callback.json.gz 2026-03-24 17:00:09,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:09,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c16778148209fd5e05 HTTP/1.1" 200 None 2026-03-24 17:00:09,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:09,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:09,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:09,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c16778148209fd5e05 HTTP/1.1" 200 None 2026-03-24 17:00:09,794 - lava_callback - INFO - Completed processing callback for node 69c2c2c16778148209fd5e05 2026-03-24 17:00:10,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:10,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:10,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b46778148209fd5de4/lava-logs.yaml 2026-03-24 17:00:10,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:11,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:11,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b46778148209fd5de4/lava_callback.json.gz 2026-03-24 17:00:11,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:11,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b46778148209fd5de4 HTTP/1.1" 200 None 2026-03-24 17:00:11,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:11,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:11,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:11,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b46778148209fd5de4 HTTP/1.1" 200 None 2026-03-24 17:00:11,492 - lava_callback - INFO - Completed processing callback for node 69c2c2b46778148209fd5de4 2026-03-24 17:00:13,702 - lava_callback - INFO - Received callback for node 69c2c2b16778148209fd5ddb INFO: 172.21.0.1:40676 - "POST /node/69c2c2b16778148209fd5ddb HTTP/1.0" 202 Accepted 2026-03-24 17:00:13,715 - lava_callback - INFO - Starting async job submit for node 69c2c2b16778148209fd5ddb 2026-03-24 17:00:13,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:13,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b16778148209fd5ddb HTTP/1.1" 200 None 2026-03-24 17:00:13,787 - lava_callback - DEBUG - Processing node 69c2c2b16778148209fd5ddb: baseline-arm64 2026-03-24 17:00:14,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:14,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:14,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b16778148209fd5ddb/log.txt.gz 2026-03-24 17:00:14,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40692 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 17:00:15,278 - lava_callback - INFO - Received callback for node 69c2c2b66778148209fd5dea INFO: 172.21.0.1:40696 - "POST /node/69c2c2b66778148209fd5dea HTTP/1.0" 202 Accepted 2026-03-24 17:00:15,288 - lava_callback - INFO - Starting async job submit for node 69c2c2b66778148209fd5dea 2026-03-24 17:00:15,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:15,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b16778148209fd5ddb/lava-logs.yaml 2026-03-24 17:00:15,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:15,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:15,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b66778148209fd5dea HTTP/1.1" 200 None 2026-03-24 17:00:15,366 - lava_callback - DEBUG - Processing node 69c2c2b66778148209fd5dea: baseline-arm64 2026-03-24 17:00:15,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:15,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b16778148209fd5ddb/lava_callback.json.gz 2026-03-24 17:00:15,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:15,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:15,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b16778148209fd5ddb HTTP/1.1" 200 None 2026-03-24 17:00:15,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:15,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:15,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:15,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b16778148209fd5ddb HTTP/1.1" 200 None 2026-03-24 17:00:15,957 - lava_callback - INFO - Completed processing callback for node 69c2c2b16778148209fd5ddb 2026-03-24 17:00:16,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:16,214 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b66778148209fd5dea/log.txt.gz 2026-03-24 17:00:16,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40700 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:00:16,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:16,972 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b66778148209fd5dea/lava-logs.yaml 2026-03-24 17:00:16,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:17,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:17,391 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b66778148209fd5dea/lava_callback.json.gz 2026-03-24 17:00:17,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:17,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b66778148209fd5dea HTTP/1.1" 200 None 2026-03-24 17:00:17,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:17,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:17,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:17,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b66778148209fd5dea HTTP/1.1" 200 None 2026-03-24 17:00:17,537 - lava_callback - INFO - Completed processing callback for node 69c2c2b66778148209fd5dea 2026-03-24 17:00:23,188 - lava_callback - INFO - Received callback for node 69c2c2c36778148209fd5e0b INFO: 172.21.0.1:51904 - "POST /node/69c2c2c36778148209fd5e0b HTTP/1.0" 202 Accepted 2026-03-24 17:00:23,201 - lava_callback - INFO - Starting async job submit for node 69c2c2c36778148209fd5e0b 2026-03-24 17:00:23,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:23,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c36778148209fd5e0b HTTP/1.1" 200 None 2026-03-24 17:00:23,270 - lava_callback - DEBUG - Processing node 69c2c2c36778148209fd5e0b: baseline-nfs-arm64 INFO: 172.21.0.1:51918 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:00:23,729 - lava_callback - INFO - Received callback for node 69c2c2c96778148209fd5e3e INFO: 172.21.0.1:51922 - "POST /node/69c2c2c96778148209fd5e3e HTTP/1.0" 202 Accepted 2026-03-24 17:00:23,755 - lava_callback - INFO - Starting async job submit for node 69c2c2c96778148209fd5e3e 2026-03-24 17:00:23,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:23,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c96778148209fd5e3e HTTP/1.1" 200 None 2026-03-24 17:00:23,935 - lava_callback - DEBUG - Processing node 69c2c2c96778148209fd5e3e: baseline-nfs-arm64 2026-03-24 17:00:24,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:24,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:24,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:24,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c36778148209fd5e0b/log.txt.gz 2026-03-24 17:00:25,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:25,064 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c96778148209fd5e3e/log.txt.gz 2026-03-24 17:00:25,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:25,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:26,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:26,108 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c36778148209fd5e0b/lava-logs.yaml 2026-03-24 17:00:26,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:26,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:26,138 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c96778148209fd5e3e/lava-logs.yaml 2026-03-24 17:00:26,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:26,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:26,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c36778148209fd5e0b/lava_callback.json.gz INFO: 172.21.0.1:51924 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:00:26,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:26,558 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c96778148209fd5e3e/lava_callback.json.gz 2026-03-24 17:00:26,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:26,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:26,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c36778148209fd5e0b HTTP/1.1" 200 None 2026-03-24 17:00:26,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:26,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c96778148209fd5e3e HTTP/1.1" 200 None 2026-03-24 17:00:26,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:26,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:26,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:26,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:26,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:26,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c36778148209fd5e0b HTTP/1.1" 200 None 2026-03-24 17:00:26,747 - lava_callback - INFO - Completed processing callback for node 69c2c2c36778148209fd5e0b 2026-03-24 17:00:26,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c96778148209fd5e3e HTTP/1.1" 200 None 2026-03-24 17:00:26,849 - lava_callback - INFO - Completed processing callback for node 69c2c2c96778148209fd5e3e 2026-03-24 17:00:32,643 - lava_callback - INFO - Received callback for node 69c2c2f86778148209fd5ecd INFO: 172.21.0.1:40284 - "POST /node/69c2c2f86778148209fd5ecd HTTP/1.0" 202 Accepted 2026-03-24 17:00:32,664 - lava_callback - INFO - Starting async job submit for node 69c2c2f86778148209fd5ecd 2026-03-24 17:00:32,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:32,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f86778148209fd5ecd HTTP/1.1" 200 None 2026-03-24 17:00:32,729 - lava_callback - DEBUG - Processing node 69c2c2f86778148209fd5ecd: baseline-arm64-foundriesio 2026-03-24 17:00:33,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:33,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:33,438 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c2f86778148209fd5ecd/log.txt.gz 2026-03-24 17:00:33,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:34,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:34,159 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c2f86778148209fd5ecd/lava-logs.yaml 2026-03-24 17:00:34,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:34,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:34,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 17:00:34,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:34,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c2f86778148209fd5ecd/lava_callback.json.gz 2026-03-24 17:00:34,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:34,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f86778148209fd5ecd HTTP/1.1" 200 None 2026-03-24 17:00:34,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:34,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:34,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:34,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2f86778148209fd5ecd HTTP/1.1" 200 None 2026-03-24 17:00:34,659 - lava_callback - INFO - Completed processing callback for node 69c2c2f86778148209fd5ecd INFO: 172.21.0.1:40290 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:00:36,494 - lava_callback - INFO - Received callback for node 69c02027c3d73288b0d07e4a INFO: 172.21.0.1:40292 - "POST /node/69c02027c3d73288b0d07e4a HTTP/1.0" 202 Accepted 2026-03-24 17:00:36,503 - lava_callback - INFO - Starting async job submit for node 69c02027c3d73288b0d07e4a 2026-03-24 17:00:36,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:36,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02027c3d73288b0d07e4a HTTP/1.1" 200 None 2026-03-24 17:00:36,541 - lava_callback - DEBUG - Processing node 69c02027c3d73288b0d07e4a: baseline-arm64-clabbe 2026-03-24 17:00:36,541 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02027c3d73288b0d07e4a 2026-03-24 17:00:36,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:36,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02027c3d73288b0d07e4a HTTP/1.1" 200 None 2026-03-24 17:00:36,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:36,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 17:00:36,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:36,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02027c3d73288b0d07e4a HTTP/1.1" 200 None 2026-03-24 17:00:36,643 - lava_callback - ERROR - Error processing callback for node 69c02027c3d73288b0d07e4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:00:38,488 - lava_callback - INFO - Received callback for node 69c2c2f46778148209fd5ec0 INFO: 172.21.0.1:40308 - "POST /node/69c2c2f46778148209fd5ec0 HTTP/1.0" 202 Accepted 2026-03-24 17:00:38,499 - lava_callback - INFO - Starting async job submit for node 69c2c2f46778148209fd5ec0 2026-03-24 17:00:38,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:38,543 - lava_callback - INFO - Received callback for node 69c2c2c56778148209fd5e0e INFO: 172.21.0.1:40316 - "POST /node/69c2c2c56778148209fd5e0e HTTP/1.0" 202 Accepted 2026-03-24 17:00:38,567 - lava_callback - INFO - Starting async job submit for node 69c2c2c56778148209fd5e0e 2026-03-24 17:00:38,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f46778148209fd5ec0 HTTP/1.1" 200 None 2026-03-24 17:00:38,633 - lava_callback - DEBUG - Processing node 69c2c2f46778148209fd5ec0: baseline-arm64-pengutronix 2026-03-24 17:00:38,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:38,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c56778148209fd5e0e HTTP/1.1" 200 None 2026-03-24 17:00:38,727 - lava_callback - DEBUG - Processing node 69c2c2c56778148209fd5e0e: baseline-nfs-arm64 2026-03-24 17:00:39,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:39,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:39,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f46778148209fd5ec0/log.txt.gz 2026-03-24 17:00:40,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:40,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:40,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:40,957 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f46778148209fd5ec0/lava-logs.yaml 2026-03-24 17:00:40,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:41,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:41,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c56778148209fd5e0e/log.txt.gz 2026-03-24 17:00:41,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:41,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f46778148209fd5ec0/lava_callback.json.gz 2026-03-24 17:00:41,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:41,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f46778148209fd5ec0 HTTP/1.1" 200 None 2026-03-24 17:00:41,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:41,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:41,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:41,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2f46778148209fd5ec0 HTTP/1.1" 200 None 2026-03-24 17:00:41,715 - lava_callback - INFO - Completed processing callback for node 69c2c2f46778148209fd5ec0 2026-03-24 17:00:41,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:42,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:42,683 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c56778148209fd5e0e/lava-logs.yaml 2026-03-24 17:00:42,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:42,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:42,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c56778148209fd5e0e/lava_callback.json.gz 2026-03-24 17:00:42,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:42,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c56778148209fd5e0e HTTP/1.1" 200 None 2026-03-24 17:00:42,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:42,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:42,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:42,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c56778148209fd5e0e HTTP/1.1" 200 None 2026-03-24 17:00:43,001 - lava_callback - INFO - Completed processing callback for node 69c2c2c56778148209fd5e0e 2026-03-24 17:00:43,497 - lava_callback - INFO - Received callback for node 69c2c2b96778148209fd5df0 INFO: 172.21.0.1:43704 - "POST /node/69c2c2b96778148209fd5df0 HTTP/1.0" 202 Accepted 2026-03-24 17:00:43,503 - lava_callback - INFO - Starting async job submit for node 69c2c2b96778148209fd5df0 2026-03-24 17:00:43,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:43,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b96778148209fd5df0 HTTP/1.1" 200 None 2026-03-24 17:00:43,552 - lava_callback - DEBUG - Processing node 69c2c2b96778148209fd5df0: baseline-arm64 2026-03-24 17:00:43,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:43,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:43,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b96778148209fd5df0/log.txt.gz 2026-03-24 17:00:44,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:44,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:44,974 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b96778148209fd5df0/lava-logs.yaml 2026-03-24 17:00:44,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:45,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:45,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b96778148209fd5df0/lava_callback.json.gz 2026-03-24 17:00:45,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:45,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b96778148209fd5df0 HTTP/1.1" 200 None 2026-03-24 17:00:45,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:45,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:45,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:45,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b96778148209fd5df0 HTTP/1.1" 200 None 2026-03-24 17:00:45,613 - lava_callback - INFO - Completed processing callback for node 69c2c2b96778148209fd5df0 2026-03-24 17:00:46,138 - lava_callback - INFO - Received callback for node 69c2c2f56778148209fd5ec3 INFO: 172.21.0.1:43714 - "POST /node/69c2c2f56778148209fd5ec3 HTTP/1.0" 202 Accepted 2026-03-24 17:00:46,148 - lava_callback - INFO - Starting async job submit for node 69c2c2f56778148209fd5ec3 2026-03-24 17:00:46,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:46,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f56778148209fd5ec3 HTTP/1.1" 200 None 2026-03-24 17:00:46,221 - lava_callback - DEBUG - Processing node 69c2c2f56778148209fd5ec3: baseline-arm64-pengutronix INFO: 172.21.0.1:43718 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:00:46,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:46,639 - lava_callback - INFO - Received callback for node 69c2c2c76778148209fd5e38 INFO: 172.21.0.1:43724 - "POST /node/69c2c2c76778148209fd5e38 HTTP/1.0" 202 Accepted 2026-03-24 17:00:46,653 - lava_callback - INFO - Starting async job submit for node 69c2c2c76778148209fd5e38 2026-03-24 17:00:46,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:46,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c76778148209fd5e38 HTTP/1.1" 200 None 2026-03-24 17:00:46,734 - lava_callback - DEBUG - Processing node 69c2c2c76778148209fd5e38: baseline-nfs-arm64 2026-03-24 17:00:46,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:46,943 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f56778148209fd5ec3/log.txt.gz 2026-03-24 17:00:47,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:47,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:47,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:47,829 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c76778148209fd5e38/log.txt.gz 2026-03-24 17:00:47,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:47,981 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f56778148209fd5ec3/lava-logs.yaml 2026-03-24 17:00:48,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:48,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:48,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:48,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f56778148209fd5ec3/lava_callback.json.gz 2026-03-24 17:00:48,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:48,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f56778148209fd5ec3 HTTP/1.1" 200 None 2026-03-24 17:00:48,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:48,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:48,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:48,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2f56778148209fd5ec3 HTTP/1.1" 200 None 2026-03-24 17:00:48,537 - lava_callback - INFO - Completed processing callback for node 69c2c2f56778148209fd5ec3 2026-03-24 17:00:48,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:48,828 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c76778148209fd5e38/lava-logs.yaml 2026-03-24 17:00:48,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:49,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:49,158 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c76778148209fd5e38/lava_callback.json.gz 2026-03-24 17:00:49,197 - lava_callback - INFO - Received callback for node 69c2c2cb6778148209fd5e41 INFO: 172.21.0.1:43728 - "POST /node/69c2c2cb6778148209fd5e41 HTTP/1.0" 202 Accepted 2026-03-24 17:00:49,219 - lava_callback - INFO - Starting async job submit for node 69c2c2cb6778148209fd5e41 2026-03-24 17:00:49,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:49,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:49,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c76778148209fd5e38 HTTP/1.1" 200 None 2026-03-24 17:00:49,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:49,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:49,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:49,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2cb6778148209fd5e41 HTTP/1.1" 200 None 2026-03-24 17:00:49,322 - lava_callback - DEBUG - Processing node 69c2c2cb6778148209fd5e41: baseline-nfs-arm64 2026-03-24 17:00:49,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c76778148209fd5e38 HTTP/1.1" 200 None 2026-03-24 17:00:49,401 - lava_callback - INFO - Completed processing callback for node 69c2c2c76778148209fd5e38 2026-03-24 17:00:50,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:50,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:50,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2cb6778148209fd5e41/log.txt.gz 2026-03-24 17:00:50,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:51,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:51,254 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2cb6778148209fd5e41/lava-logs.yaml 2026-03-24 17:00:51,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:00:51,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:00:51,597 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2cb6778148209fd5e41/lava_callback.json.gz 2026-03-24 17:00:51,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:51,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2cb6778148209fd5e41 HTTP/1.1" 200 None 2026-03-24 17:00:51,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:51,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:00:51,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:00:51,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2cb6778148209fd5e41 HTTP/1.1" 200 None 2026-03-24 17:00:51,752 - lava_callback - INFO - Completed processing callback for node 69c2c2cb6778148209fd5e41 INFO: 172.21.0.1:58970 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:01:00,012 - lava_callback - INFO - Received callback for node 69c2c2f66778148209fd5eca INFO: 172.21.0.1:41306 - "POST /node/69c2c2f66778148209fd5eca HTTP/1.0" 202 Accepted 2026-03-24 17:01:00,023 - lava_callback - INFO - Starting async job submit for node 69c2c2f66778148209fd5eca 2026-03-24 17:01:00,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:00,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f66778148209fd5eca HTTP/1.1" 200 None 2026-03-24 17:01:00,095 - lava_callback - DEBUG - Processing node 69c2c2f66778148209fd5eca: baseline-arm64-pengutronix 2026-03-24 17:01:00,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:01,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:01,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f66778148209fd5eca/log.txt.gz 2026-03-24 17:01:01,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:02,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:02,372 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f66778148209fd5eca/lava-logs.yaml 2026-03-24 17:01:02,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:03,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:03,229 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2c2f66778148209fd5eca/lava_callback.json.gz 2026-03-24 17:01:03,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:03,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f66778148209fd5eca HTTP/1.1" 200 None 2026-03-24 17:01:03,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:03,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:01:03,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:03,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2f66778148209fd5eca HTTP/1.1" 200 None 2026-03-24 17:01:03,375 - lava_callback - INFO - Completed processing callback for node 69c2c2f66778148209fd5eca 2026-03-24 17:01:04,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:04,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 17:01:04,577 - lava_callback - INFO - Received callback for node 69c2c2ba6778148209fd5df3 INFO: 172.21.0.1:41314 - "POST /node/69c2c2ba6778148209fd5df3 HTTP/1.0" 202 Accepted 2026-03-24 17:01:04,587 - lava_callback - INFO - Starting async job submit for node 69c2c2ba6778148209fd5df3 2026-03-24 17:01:04,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:04,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ba6778148209fd5df3 HTTP/1.1" 200 None 2026-03-24 17:01:04,727 - lava_callback - DEBUG - Processing node 69c2c2ba6778148209fd5df3: baseline-arm64 2026-03-24 17:01:05,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:05,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:05,641 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ba6778148209fd5df3/log.txt.gz 2026-03-24 17:01:05,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:06,082 - lava_callback - INFO - Received callback for node 69c2c2be6778148209fd5dfe INFO: 172.21.0.1:41318 - "POST /node/69c2c2be6778148209fd5dfe HTTP/1.0" 202 Accepted 2026-03-24 17:01:06,096 - lava_callback - INFO - Starting async job submit for node 69c2c2be6778148209fd5dfe 2026-03-24 17:01:06,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:06,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2be6778148209fd5dfe HTTP/1.1" 200 None 2026-03-24 17:01:06,170 - lava_callback - DEBUG - Processing node 69c2c2be6778148209fd5dfe: baseline-nfs-arm64 2026-03-24 17:01:06,434 - lava_callback - INFO - Received callback for node 69c2c2e86778148209fd5ea5 INFO: 172.21.0.1:41328 - "POST /node/69c2c2e86778148209fd5ea5 HTTP/1.0" 202 Accepted 2026-03-24 17:01:06,459 - lava_callback - INFO - Starting async job submit for node 69c2c2e86778148209fd5ea5 INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:01:06,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:06,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e86778148209fd5ea5 HTTP/1.1" 200 None 2026-03-24 17:01:06,612 - lava_callback - DEBUG - Processing node 69c2c2e86778148209fd5ea5: baseline-arm64-broonie 2026-03-24 17:01:07,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:07,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:07,112 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ba6778148209fd5df3/lava-logs.yaml 2026-03-24 17:01:07,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:07,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:07,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:07,732 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2be6778148209fd5dfe/log.txt.gz 2026-03-24 17:01:07,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:07,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e86778148209fd5ea5/log.txt.gz 2026-03-24 17:01:08,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:08,064 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2ba6778148209fd5df3/lava_callback.json.gz 2026-03-24 17:01:08,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:08,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:08,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ba6778148209fd5df3 HTTP/1.1" 200 None 2026-03-24 17:01:08,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:08,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:01:08,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:08,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:08,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ba6778148209fd5df3 HTTP/1.1" 200 None 2026-03-24 17:01:08,464 - lava_callback - INFO - Completed processing callback for node 69c2c2ba6778148209fd5df3 2026-03-24 17:01:09,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:09,732 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2be6778148209fd5dfe/lava-logs.yaml 2026-03-24 17:01:09,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:09,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:09,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e86778148209fd5ea5/lava-logs.yaml 2026-03-24 17:01:09,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:01:10,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:10,563 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e86778148209fd5ea5/lava_callback.json.gz 2026-03-24 17:01:10,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:10,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:01:10,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2be6778148209fd5dfe/lava_callback.json.gz 2026-03-24 17:01:10,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e86778148209fd5ea5 HTTP/1.1" 200 None 2026-03-24 17:01:10,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:10,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:10,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:01:10,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:10,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2be6778148209fd5dfe HTTP/1.1" 200 None 2026-03-24 17:01:10,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:10,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2e86778148209fd5ea5 HTTP/1.1" 200 None 2026-03-24 17:01:10,767 - lava_callback - INFO - Completed processing callback for node 69c2c2e86778148209fd5ea5 2026-03-24 17:01:10,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:01:10,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:10,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2be6778148209fd5dfe HTTP/1.1" 200 None 2026-03-24 17:01:10,912 - lava_callback - INFO - Completed processing callback for node 69c2c2be6778148209fd5dfe INFO: 172.21.0.1:33080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33690 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:01:34,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:34,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:01:46,126 - lava_callback - INFO - Received callback for node 69c0206cc3d73288b0d07e59 INFO: 172.21.0.1:51092 - "POST /node/69c0206cc3d73288b0d07e59 HTTP/1.0" 202 Accepted 2026-03-24 17:01:46,135 - lava_callback - INFO - Starting async job submit for node 69c0206cc3d73288b0d07e59 2026-03-24 17:01:46,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:46,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0206cc3d73288b0d07e59 HTTP/1.1" 200 None 2026-03-24 17:01:46,196 - lava_callback - DEBUG - Processing node 69c0206cc3d73288b0d07e59: baseline-arm64-baylibre 2026-03-24 17:01:46,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0206cc3d73288b0d07e59 2026-03-24 17:01:46,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:46,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0206cc3d73288b0d07e59 HTTP/1.1" 200 None 2026-03-24 17:01:46,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:46,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 17:01:46,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:01:46,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0206cc3d73288b0d07e59 HTTP/1.1" 200 None 2026-03-24 17:01:46,310 - lava_callback - ERROR - Error processing callback for node 69c0206cc3d73288b0d07e59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53280 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:02:02,204 - lava_callback - INFO - Received callback for node 69c2c2af6778148209fd5dd8 INFO: 172.21.0.1:45624 - "POST /node/69c2c2af6778148209fd5dd8 HTTP/1.0" 202 Accepted 2026-03-24 17:02:02,213 - lava_callback - INFO - Starting async job submit for node 69c2c2af6778148209fd5dd8 2026-03-24 17:02:02,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:02:02,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2af6778148209fd5dd8 HTTP/1.1" 200 None 2026-03-24 17:02:02,265 - lava_callback - DEBUG - Processing node 69c2c2af6778148209fd5dd8: baseline-arm64 2026-03-24 17:02:02,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:02:04,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:02:04,249 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2af6778148209fd5dd8/log.txt.gz 2026-03-24 17:02:04,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:02:05,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:02:05,959 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2af6778148209fd5dd8/lava-logs.yaml 2026-03-24 17:02:05,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:02:07,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:02:07,400 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2af6778148209fd5dd8/lava_callback.json.gz 2026-03-24 17:02:07,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:02:07,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2af6778148209fd5dd8 HTTP/1.1" 200 None 2026-03-24 17:02:07,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:02:07,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:02:07,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:02:07,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2af6778148209fd5dd8 HTTP/1.1" 200 None 2026-03-24 17:02:07,537 - lava_callback - INFO - Completed processing callback for node 69c2c2af6778148209fd5dd8 INFO: 172.21.0.1:34806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54834 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:02:34,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:02:34,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60414 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:03:07,882 - lava_callback - INFO - Received callback for node 69c2c2bf6778148209fd5e02 INFO: 172.21.0.1:60426 - "POST /node/69c2c2bf6778148209fd5e02 HTTP/1.0" 202 Accepted 2026-03-24 17:03:07,892 - lava_callback - INFO - Starting async job submit for node 69c2c2bf6778148209fd5e02 2026-03-24 17:03:07,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:07,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bf6778148209fd5e02 HTTP/1.1" 200 None 2026-03-24 17:03:07,945 - lava_callback - DEBUG - Processing node 69c2c2bf6778148209fd5e02: baseline-nfs-arm64 2026-03-24 17:03:08,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:03:10,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:03:10,092 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2bf6778148209fd5e02/log.txt.gz 2026-03-24 17:03:10,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:03:12,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:03:12,006 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2bf6778148209fd5e02/lava-logs.yaml 2026-03-24 17:03:12,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:03:13,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:03:13,532 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2bf6778148209fd5e02/lava_callback.json.gz 2026-03-24 17:03:13,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:13,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bf6778148209fd5e02 HTTP/1.1" 200 None 2026-03-24 17:03:13,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:13,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:03:13,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:13,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2bf6778148209fd5e02 HTTP/1.1" 200 None 2026-03-24 17:03:13,676 - lava_callback - INFO - Completed processing callback for node 69c2c2bf6778148209fd5e02 INFO: 172.21.0.1:39722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:03:34,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:34,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46028 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:03:36,777 - lava_callback - INFO - Received callback for node 69c020dfc3d73288b0d07e7c INFO: 172.21.0.1:46038 - "POST /node/69c020dfc3d73288b0d07e7c HTTP/1.0" 202 Accepted 2026-03-24 17:03:36,785 - lava_callback - INFO - Starting async job submit for node 69c020dfc3d73288b0d07e7c 2026-03-24 17:03:36,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:36,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c020dfc3d73288b0d07e7c HTTP/1.1" 200 None 2026-03-24 17:03:36,818 - lava_callback - DEBUG - Processing node 69c020dfc3d73288b0d07e7c: baseline-arm64-clabbe 2026-03-24 17:03:36,818 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c020dfc3d73288b0d07e7c 2026-03-24 17:03:36,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:36,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c020dfc3d73288b0d07e7c HTTP/1.1" 200 None 2026-03-24 17:03:36,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:36,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 17:03:36,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:03:36,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c020dfc3d73288b0d07e7c HTTP/1.1" 200 None 2026-03-24 17:03:36,885 - lava_callback - ERROR - Error processing callback for node 69c020dfc3d73288b0d07e7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33564 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:04:31,725 - lava_callback - INFO - Received callback for node 69c2c2f96778148209fd5ed2 INFO: 172.21.0.1:57126 - "POST /node/69c2c2f96778148209fd5ed2 HTTP/1.0" 202 Accepted 2026-03-24 17:04:31,741 - lava_callback - INFO - Starting async job submit for node 69c2c2f96778148209fd5ed2 2026-03-24 17:04:31,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:31,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f96778148209fd5ed2 HTTP/1.1" 200 None 2026-03-24 17:04:31,849 - lava_callback - DEBUG - Processing node 69c2c2f96778148209fd5ed2: baseline-arm64-foundriesio 2026-03-24 17:04:33,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:04:34,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:04:34,172 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c2f96778148209fd5ed2/log.txt.gz 2026-03-24 17:04:35,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:04:36,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:04:36,100 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c2f96778148209fd5ed2/lava-logs.yaml 2026-03-24 17:04:36,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:57128 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:04:36,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:04:36,646 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c2f96778148209fd5ed2/lava_callback.json.gz DEBUG: 69c2c2f96778148209fd5ed2 Transitting job node result: pass -> incomplete 2026-03-24 17:04:36,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:36,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2f96778148209fd5ed2 HTTP/1.1" 200 None 2026-03-24 17:04:36,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:36,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:04:36,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:36,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2f96778148209fd5ed2 HTTP/1.1" 200 None 2026-03-24 17:04:36,818 - lava_callback - INFO - Completed processing callback for node 69c2c2f96778148209fd5ed2 INFO: 172.21.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46508 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:04:56,750 - lava_callback - INFO - Received callback for node 69c0212bc3d73288b0d07e8b INFO: 172.21.0.1:46524 - "POST /node/69c0212bc3d73288b0d07e8b HTTP/1.0" 202 Accepted 2026-03-24 17:04:56,756 - lava_callback - INFO - Starting async job submit for node 69c0212bc3d73288b0d07e8b 2026-03-24 17:04:56,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:56,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0212bc3d73288b0d07e8b HTTP/1.1" 200 None 2026-03-24 17:04:56,774 - lava_callback - DEBUG - Processing node 69c0212bc3d73288b0d07e8b: baseline-arm64-clabbe 2026-03-24 17:04:56,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0212bc3d73288b0d07e8b 2026-03-24 17:04:56,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:56,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0212bc3d73288b0d07e8b HTTP/1.1" 200 None 2026-03-24 17:04:56,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:56,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 17:04:56,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:04:56,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0212bc3d73288b0d07e8b HTTP/1.1" 200 None 2026-03-24 17:04:56,843 - lava_callback - ERROR - Error processing callback for node 69c0212bc3d73288b0d07e8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:05:04,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:05:04,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53644 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:06:00,020 - lava_callback - INFO - Received callback for node 69c2c2bd6778148209fd5df9 INFO: 172.21.0.1:56350 - "POST /node/69c2c2bd6778148209fd5df9 HTTP/1.0" 202 Accepted 2026-03-24 17:06:00,032 - lava_callback - INFO - Starting async job submit for node 69c2c2bd6778148209fd5df9 2026-03-24 17:06:00,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:00,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bd6778148209fd5df9 HTTP/1.1" 200 None 2026-03-24 17:06:00,084 - lava_callback - DEBUG - Processing node 69c2c2bd6778148209fd5df9: baseline-arm64 2026-03-24 17:06:00,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:06:00,622 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:06:00,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2bd6778148209fd5df9/log.txt.gz 2026-03-24 17:06:00,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:06:01,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:06:01,070 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2bd6778148209fd5df9/lava-logs.yaml 2026-03-24 17:06:01,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:06:01,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:06:01,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2bd6778148209fd5df9/lava_callback.json.gz DEBUG: Setup failed for 69c2c2bd6778148209fd5df9. Transitting job node result: incomplete -> fail 2026-03-24 17:06:01,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:01,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bd6778148209fd5df9 HTTP/1.1" 200 None 2026-03-24 17:06:01,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:01,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:06:01,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:01,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2bd6778148209fd5df9 HTTP/1.1" 200 None 2026-03-24 17:06:01,485 - lava_callback - INFO - Completed processing callback for node 69c2c2bd6778148209fd5df9 2026-03-24 17:06:04,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:04,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:06:16,731 - lava_callback - INFO - Received callback for node 69c02177c3d73288b0d07e9d INFO: 172.21.0.1:41976 - "POST /node/69c02177c3d73288b0d07e9d HTTP/1.0" 202 Accepted 2026-03-24 17:06:16,737 - lava_callback - INFO - Starting async job submit for node 69c02177c3d73288b0d07e9d 2026-03-24 17:06:16,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:16,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02177c3d73288b0d07e9d HTTP/1.1" 200 None 2026-03-24 17:06:16,754 - lava_callback - DEBUG - Processing node 69c02177c3d73288b0d07e9d: baseline-arm64-clabbe 2026-03-24 17:06:16,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02177c3d73288b0d07e9d 2026-03-24 17:06:16,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:16,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02177c3d73288b0d07e9d HTTP/1.1" 200 None 2026-03-24 17:06:16,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:16,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 17:06:16,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:16,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02177c3d73288b0d07e9d HTTP/1.1" 200 None 2026-03-24 17:06:16,830 - lava_callback - ERROR - Error processing callback for node 69c02177c3d73288b0d07e9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:06:20,809 - lava_callback - INFO - Received callback for node 69c2c4686778148209fd6002 INFO: 172.21.0.1:40116 - "POST /node/69c2c4686778148209fd6002 HTTP/1.0" 202 Accepted 2026-03-24 17:06:20,815 - lava_callback - INFO - Starting async job submit for node 69c2c4686778148209fd6002 2026-03-24 17:06:20,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:20,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c4686778148209fd6002 HTTP/1.1" 200 None 2026-03-24 17:06:20,833 - lava_callback - DEBUG - Processing node 69c2c4686778148209fd6002: baseline-arm 2026-03-24 17:06:20,838 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:06:20,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:06:21,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:06:21,019 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2c4686778148209fd6002/lava-logs.yaml 2026-03-24 17:06:21,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:06:21,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:06:21,157 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2c4686778148209fd6002/lava_callback.json.gz 2026-03-24 17:06:21,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:21,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c4686778148209fd6002 HTTP/1.1" 200 None 2026-03-24 17:06:21,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:21,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 17:06:21,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:21,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c4686778148209fd6002 HTTP/1.1" 200 None 2026-03-24 17:06:21,229 - lava_callback - INFO - Completed processing callback for node 69c2c4686778148209fd6002 INFO: 172.21.0.1:40128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40138 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:06:34,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:06:34,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40050 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:07:37,051 - lava_callback - INFO - Received callback for node 69c021c2c3d73288b0d07eab INFO: 172.21.0.1:40064 - "POST /node/69c021c2c3d73288b0d07eab HTTP/1.0" 202 Accepted 2026-03-24 17:07:37,060 - lava_callback - INFO - Starting async job submit for node 69c021c2c3d73288b0d07eab 2026-03-24 17:07:37,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:07:37,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c021c2c3d73288b0d07eab HTTP/1.1" 200 None 2026-03-24 17:07:37,087 - lava_callback - DEBUG - Processing node 69c021c2c3d73288b0d07eab: baseline-arm64-clabbe 2026-03-24 17:07:37,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c021c2c3d73288b0d07eab 2026-03-24 17:07:37,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:07:37,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c021c2c3d73288b0d07eab HTTP/1.1" 200 None 2026-03-24 17:07:37,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:07:37,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 17:07:37,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:07:37,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c021c2c3d73288b0d07eab HTTP/1.1" 200 None 2026-03-24 17:07:37,208 - lava_callback - ERROR - Error processing callback for node 69c021c2c3d73288b0d07eab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43006 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:08:36,941 - lava_callback - INFO - Received callback for node 69c0220dc3d73288b0d07eb3 INFO: 172.21.0.1:43012 - "POST /node/69c0220dc3d73288b0d07eb3 HTTP/1.0" 202 Accepted 2026-03-24 17:08:36,950 - lava_callback - INFO - Starting async job submit for node 69c0220dc3d73288b0d07eb3 2026-03-24 17:08:36,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:08:36,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0220dc3d73288b0d07eb3 HTTP/1.1" 200 None 2026-03-24 17:08:36,970 - lava_callback - DEBUG - Processing node 69c0220dc3d73288b0d07eb3: baseline-arm64-clabbe 2026-03-24 17:08:36,970 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0220dc3d73288b0d07eb3 2026-03-24 17:08:36,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:08:36,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0220dc3d73288b0d07eb3 HTTP/1.1" 200 None 2026-03-24 17:08:36,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:08:36,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 17:08:37,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:08:37,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0220dc3d73288b0d07eb3 HTTP/1.1" 200 None 2026-03-24 17:08:37,049 - lava_callback - ERROR - Error processing callback for node 69c0220dc3d73288b0d07eb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:09:32,106 - lava_callback - INFO - Received callback for node 69c2c2b86778148209fd5ded INFO: 172.21.0.1:40932 - "POST /node/69c2c2b86778148209fd5ded HTTP/1.0" 202 Accepted 2026-03-24 17:09:32,113 - lava_callback - INFO - Starting async job submit for node 69c2c2b86778148209fd5ded 2026-03-24 17:09:32,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:32,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b86778148209fd5ded HTTP/1.1" 200 None 2026-03-24 17:09:32,152 - lava_callback - DEBUG - Processing node 69c2c2b86778148209fd5ded: baseline-arm64 2026-03-24 17:09:32,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:09:32,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:09:32,835 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b86778148209fd5ded/log.txt.gz 2026-03-24 17:09:33,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:09:33,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:09:33,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b86778148209fd5ded/lava-logs.yaml 2026-03-24 17:09:33,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:09:33,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:09:33,471 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2b86778148209fd5ded/lava_callback.json.gz 2026-03-24 17:09:33,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:33,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2b86778148209fd5ded HTTP/1.1" 200 None 2026-03-24 17:09:33,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:33,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:09:33,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:33,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2b86778148209fd5ded HTTP/1.1" 200 None 2026-03-24 17:09:33,653 - lava_callback - INFO - Completed processing callback for node 69c2c2b86778148209fd5ded 2026-03-24 17:09:34,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:34,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40936 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:09:37,511 - lava_callback - INFO - Received callback for node 69c2c53a6778148209fd603a INFO: 172.21.0.1:40952 - "POST /node/69c2c53a6778148209fd603a HTTP/1.0" 202 Accepted 2026-03-24 17:09:37,517 - lava_callback - INFO - Starting async job submit for node 69c2c53a6778148209fd603a 2026-03-24 17:09:37,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:37,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c53a6778148209fd603a HTTP/1.1" 200 None 2026-03-24 17:09:37,536 - lava_callback - DEBUG - Processing node 69c2c53a6778148209fd603a: baseline-nfs-arm64 2026-03-24 17:09:37,537 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:09:37,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:09:37,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:09:37,652 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c53a6778148209fd603a/lava-logs.yaml 2026-03-24 17:09:37,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:09:37,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:09:37,765 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c53a6778148209fd603a/lava_callback.json.gz 2026-03-24 17:09:37,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:37,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c53a6778148209fd603a HTTP/1.1" 200 None 2026-03-24 17:09:37,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:37,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:09:37,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:37,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c53a6778148209fd603a HTTP/1.1" 200 None 2026-03-24 17:09:37,824 - lava_callback - INFO - Completed processing callback for node 69c2c53a6778148209fd603a INFO: 172.21.0.1:46186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41392 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:09:57,030 - lava_callback - INFO - Received callback for node 69c02257c3d73288b0d07ec1 INFO: 172.21.0.1:41398 - "POST /node/69c02257c3d73288b0d07ec1 HTTP/1.0" 202 Accepted 2026-03-24 17:09:57,035 - lava_callback - INFO - Starting async job submit for node 69c02257c3d73288b0d07ec1 2026-03-24 17:09:57,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:57,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02257c3d73288b0d07ec1 HTTP/1.1" 200 None 2026-03-24 17:09:57,052 - lava_callback - DEBUG - Processing node 69c02257c3d73288b0d07ec1: baseline-arm64-clabbe 2026-03-24 17:09:57,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02257c3d73288b0d07ec1 2026-03-24 17:09:57,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:57,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02257c3d73288b0d07ec1 HTTP/1.1" 200 None 2026-03-24 17:09:57,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:57,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 17:09:57,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:09:57,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02257c3d73288b0d07ec1 HTTP/1.1" 200 None 2026-03-24 17:09:57,235 - lava_callback - ERROR - Error processing callback for node 69c02257c3d73288b0d07ec1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:10:04,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:10:04,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41376 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:10:17,402 - lava_callback - INFO - Received callback for node 69c2c2c86778148209fd5e3b INFO: 172.21.0.1:41392 - "POST /node/69c2c2c86778148209fd5e3b HTTP/1.0" 202 Accepted 2026-03-24 17:10:17,409 - lava_callback - INFO - Starting async job submit for node 69c2c2c86778148209fd5e3b 2026-03-24 17:10:17,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:10:17,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c86778148209fd5e3b HTTP/1.1" 200 None 2026-03-24 17:10:17,450 - lava_callback - DEBUG - Processing node 69c2c2c86778148209fd5e3b: baseline-nfs-arm64 2026-03-24 17:10:17,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:10:18,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:10:18,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c86778148209fd5e3b/log.txt.gz 2026-03-24 17:10:18,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:10:18,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:10:18,630 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c86778148209fd5e3b/lava-logs.yaml 2026-03-24 17:10:18,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:10:18,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:10:18,814 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c86778148209fd5e3b/lava_callback.json.gz 2026-03-24 17:10:18,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:10:18,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c86778148209fd5e3b HTTP/1.1" 200 None 2026-03-24 17:10:18,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:10:18,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:10:18,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:10:18,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c86778148209fd5e3b HTTP/1.1" 200 None 2026-03-24 17:10:18,962 - lava_callback - INFO - Completed processing callback for node 69c2c2c86778148209fd5e3b INFO: 172.21.0.1:54930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54942 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:10:34,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:10:34,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:11:30,621 - lava_callback - INFO - Received callback for node 69c2c5846778148209fd604f INFO: 172.21.0.1:57524 - "POST /node/69c2c5846778148209fd604f HTTP/1.0" 202 Accepted 2026-03-24 17:11:30,628 - lava_callback - INFO - Starting async job submit for node 69c2c5846778148209fd604f 2026-03-24 17:11:30,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:11:30,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c5846778148209fd604f HTTP/1.1" 200 None 2026-03-24 17:11:30,672 - lava_callback - DEBUG - Processing node 69c2c5846778148209fd604f: baseline-arm64 2026-03-24 17:11:30,803 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:11:30,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:11:31,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:11:31,062 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c5846778148209fd604f/lava-logs.yaml 2026-03-24 17:11:31,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:11:31,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:11:31,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c5846778148209fd604f/lava_callback.json.gz 2026-03-24 17:11:31,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:11:31,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c5846778148209fd604f HTTP/1.1" 200 None 2026-03-24 17:11:31,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:11:31,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:11:31,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:11:31,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c5846778148209fd604f HTTP/1.1" 200 None 2026-03-24 17:11:31,359 - lava_callback - INFO - Completed processing callback for node 69c2c5846778148209fd604f 2026-03-24 17:11:34,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:11:34,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38052 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:12:17,194 - lava_callback - INFO - Received callback for node 69c022e8c3d73288b0d07ee4 INFO: 172.21.0.1:38068 - "POST /node/69c022e8c3d73288b0d07ee4 HTTP/1.0" 202 Accepted 2026-03-24 17:12:17,199 - lava_callback - INFO - Starting async job submit for node 69c022e8c3d73288b0d07ee4 2026-03-24 17:12:17,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:17,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c022e8c3d73288b0d07ee4 HTTP/1.1" 200 None 2026-03-24 17:12:17,217 - lava_callback - DEBUG - Processing node 69c022e8c3d73288b0d07ee4: baseline-arm64-clabbe 2026-03-24 17:12:17,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c022e8c3d73288b0d07ee4 2026-03-24 17:12:17,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:17,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c022e8c3d73288b0d07ee4 HTTP/1.1" 200 None 2026-03-24 17:12:17,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:17,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 17:12:17,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:17,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c022e8c3d73288b0d07ee4 HTTP/1.1" 200 None 2026-03-24 17:12:17,285 - lava_callback - ERROR - Error processing callback for node 69c022e8c3d73288b0d07ee4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56942 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:12:28,707 - lava_callback - INFO - Received callback for node 69c2c2e16778148209fd5e7f INFO: 172.21.0.1:56948 - "POST /node/69c2c2e16778148209fd5e7f HTTP/1.0" 202 Accepted 2026-03-24 17:12:28,714 - lava_callback - INFO - Starting async job submit for node 69c2c2e16778148209fd5e7f 2026-03-24 17:12:28,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:28,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e16778148209fd5e7f HTTP/1.1" 200 None 2026-03-24 17:12:28,751 - lava_callback - DEBUG - Processing node 69c2c2e16778148209fd5e7f: baseline-arm64-broonie 2026-03-24 17:12:29,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:12:29,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:12:29,192 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e16778148209fd5e7f/log.txt.gz 2026-03-24 17:12:29,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:12:29,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:12:29,573 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e16778148209fd5e7f/lava-logs.yaml 2026-03-24 17:12:29,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:12:29,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:12:29,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e16778148209fd5e7f/lava_callback.json.gz 2026-03-24 17:12:29,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:29,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e16778148209fd5e7f HTTP/1.1" 200 None 2026-03-24 17:12:29,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:29,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:12:29,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:29,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2e16778148209fd5e7f HTTP/1.1" 200 None 2026-03-24 17:12:29,792 - lava_callback - INFO - Completed processing callback for node 69c2c2e16778148209fd5e7f 2026-03-24 17:12:34,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:34,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47524 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:12:48,220 - lava_callback - INFO - Received callback for node 69c2c2bb6778148209fd5df6 INFO: 172.21.0.1:47538 - "POST /node/69c2c2bb6778148209fd5df6 HTTP/1.0" 202 Accepted 2026-03-24 17:12:48,227 - lava_callback - INFO - Starting async job submit for node 69c2c2bb6778148209fd5df6 2026-03-24 17:12:48,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:48,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bb6778148209fd5df6 HTTP/1.1" 200 None 2026-03-24 17:12:48,289 - lava_callback - DEBUG - Processing node 69c2c2bb6778148209fd5df6: baseline-arm64 2026-03-24 17:12:49,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:12:49,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:12:49,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2bb6778148209fd5df6/log.txt.gz 2026-03-24 17:12:49,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:12:49,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:12:49,957 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2bb6778148209fd5df6/lava-logs.yaml 2026-03-24 17:12:49,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:12:50,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:12:50,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c2bb6778148209fd5df6/lava_callback.json.gz DEBUG: Setup failed for 69c2c2bb6778148209fd5df6. Transitting job node result: incomplete -> fail 2026-03-24 17:12:50,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:50,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2bb6778148209fd5df6 HTTP/1.1" 200 None 2026-03-24 17:12:50,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:50,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:12:50,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:12:50,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2bb6778148209fd5df6 HTTP/1.1" 200 None 2026-03-24 17:12:50,427 - lava_callback - INFO - Completed processing callback for node 69c2c2bb6778148209fd5df6 INFO: 172.21.0.1:48950 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:13:04,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:04,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33998 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:13:36,788 - lava_callback - INFO - Received callback for node 69c2c5d46778148209fd6060 INFO: 172.21.0.1:34002 - "POST /node/69c2c5d46778148209fd6060 HTTP/1.0" 202 Accepted 2026-03-24 17:13:36,795 - lava_callback - INFO - Starting async job submit for node 69c2c5d46778148209fd6060 2026-03-24 17:13:36,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:36,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c5d46778148209fd6060 HTTP/1.1" 200 None 2026-03-24 17:13:36,840 - lava_callback - DEBUG - Processing node 69c2c5d46778148209fd6060: baseline-arm64 2026-03-24 17:13:37,042 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:13:37,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:37,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:37,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c5d46778148209fd6060/lava-logs.yaml 2026-03-24 17:13:37,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:37,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:37,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c5d46778148209fd6060/lava_callback.json.gz 2026-03-24 17:13:37,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:37,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c5d46778148209fd6060 HTTP/1.1" 200 None 2026-03-24 17:13:37,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:37,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:13:37,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:37,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c5d46778148209fd6060 HTTP/1.1" 200 None 2026-03-24 17:13:37,635 - lava_callback - INFO - Completed processing callback for node 69c2c5d46778148209fd6060 2026-03-24 17:13:38,523 - lava_callback - INFO - Received callback for node 69c2c2e66778148209fd5e9e INFO: 172.21.0.1:34006 - "POST /node/69c2c2e66778148209fd5e9e HTTP/1.0" 202 Accepted 2026-03-24 17:13:38,529 - lava_callback - INFO - Starting async job submit for node 69c2c2e66778148209fd5e9e 2026-03-24 17:13:38,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:38,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e66778148209fd5e9e HTTP/1.1" 200 None 2026-03-24 17:13:38,576 - lava_callback - DEBUG - Processing node 69c2c2e66778148209fd5e9e: baseline-arm64-broonie 2026-03-24 17:13:38,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:38,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:38,960 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e66778148209fd5e9e/log.txt.gz 2026-03-24 17:13:39,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:39,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:39,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e66778148209fd5e9e/lava-logs.yaml 2026-03-24 17:13:39,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:39,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:39,428 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e66778148209fd5e9e/lava_callback.json.gz 2026-03-24 17:13:39,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:39,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e66778148209fd5e9e HTTP/1.1" 200 None 2026-03-24 17:13:39,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:39,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:13:39,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:39,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2e66778148209fd5e9e HTTP/1.1" 200 None 2026-03-24 17:13:39,513 - lava_callback - INFO - Completed processing callback for node 69c2c2e66778148209fd5e9e INFO: 172.21.0.1:51558 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:13:46,664 - lava_callback - INFO - Received callback for node 69c2c2cc6778148209fd5e44 INFO: 172.21.0.1:51560 - "POST /node/69c2c2cc6778148209fd5e44 HTTP/1.0" 202 Accepted 2026-03-24 17:13:46,672 - lava_callback - INFO - Starting async job submit for node 69c2c2cc6778148209fd5e44 2026-03-24 17:13:46,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:46,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2cc6778148209fd5e44 HTTP/1.1" 200 None 2026-03-24 17:13:46,744 - lava_callback - DEBUG - Processing node 69c2c2cc6778148209fd5e44: baseline-nfs-arm64 2026-03-24 17:13:47,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:47,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:47,644 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2cc6778148209fd5e44/log.txt.gz 2026-03-24 17:13:48,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:48,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:48,316 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2cc6778148209fd5e44/lava-logs.yaml 2026-03-24 17:13:48,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:13:48,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:13:48,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2cc6778148209fd5e44/lava_callback.json.gz DEBUG: Setup failed for 69c2c2cc6778148209fd5e44. Transitting job node result: incomplete -> fail 2026-03-24 17:13:48,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:48,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2cc6778148209fd5e44 HTTP/1.1" 200 None 2026-03-24 17:13:48,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:48,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:13:48,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:13:48,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2cc6778148209fd5e44 HTTP/1.1" 200 None 2026-03-24 17:13:48,785 - lava_callback - INFO - Completed processing callback for node 69c2c2cc6778148209fd5e44 INFO: 172.21.0.1:44806 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:14:04,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:04,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48156 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:14:17,193 - lava_callback - INFO - Received callback for node 69c02359c3d73288b0d07ef9 INFO: 172.21.0.1:48166 - "POST /node/69c02359c3d73288b0d07ef9 HTTP/1.0" 202 Accepted 2026-03-24 17:14:17,199 - lava_callback - INFO - Starting async job submit for node 69c02359c3d73288b0d07ef9 2026-03-24 17:14:17,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:17,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02359c3d73288b0d07ef9 HTTP/1.1" 200 None 2026-03-24 17:14:17,217 - lava_callback - DEBUG - Processing node 69c02359c3d73288b0d07ef9: baseline-arm64-clabbe 2026-03-24 17:14:17,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02359c3d73288b0d07ef9 2026-03-24 17:14:17,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:17,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02359c3d73288b0d07ef9 HTTP/1.1" 200 None 2026-03-24 17:14:17,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:17,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 17:14:17,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:17,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02359c3d73288b0d07ef9 HTTP/1.1" 200 None 2026-03-24 17:14:17,273 - lava_callback - ERROR - Error processing callback for node 69c02359c3d73288b0d07ef9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:14:20,027 - lava_callback - INFO - Received callback for node 69c2bed26778148209fd34a3 INFO: 172.21.0.1:57706 - "POST /node/69c2bed26778148209fd34a3 HTTP/1.0" 202 Accepted 2026-03-24 17:14:20,073 - lava_callback - INFO - Starting async job submit for node 69c2bed26778148209fd34a3 2026-03-24 17:14:20,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:20,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed26778148209fd34a3 HTTP/1.1" 200 None 2026-03-24 17:14:20,124 - lava_callback - DEBUG - Processing node 69c2bed26778148209fd34a3: baseline-arm-pengutronix 2026-03-24 17:14:20,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:14:20,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:14:20,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed26778148209fd34a3/log.txt.gz 2026-03-24 17:14:20,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:14:20,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:14:20,598 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed26778148209fd34a3/lava-logs.yaml 2026-03-24 17:14:20,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:14:20,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:14:20,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2bed26778148209fd34a3/lava_callback.json.gz DEBUG: Setup failed for 69c2bed26778148209fd34a3. Transitting job node result: incomplete -> fail 2026-03-24 17:14:20,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:20,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bed26778148209fd34a3 HTTP/1.1" 200 None 2026-03-24 17:14:20,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:20,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 17:14:20,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:20,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bed26778148209fd34a3 HTTP/1.1" 200 None 2026-03-24 17:14:21,021 - lava_callback - INFO - Completed processing callback for node 69c2bed26778148209fd34a3 INFO: 172.21.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57728 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:14:29,748 - lava_callback - INFO - Received callback for node 69c2c2ef6778148209fd5eb7 INFO: 172.21.0.1:40540 - "POST /node/69c2c2ef6778148209fd5eb7 HTTP/1.0" 202 Accepted 2026-03-24 17:14:29,755 - lava_callback - INFO - Starting async job submit for node 69c2c2ef6778148209fd5eb7 2026-03-24 17:14:29,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:29,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ef6778148209fd5eb7 HTTP/1.1" 200 None 2026-03-24 17:14:29,815 - lava_callback - DEBUG - Processing node 69c2c2ef6778148209fd5eb7: baseline-arm64-clabbe 2026-03-24 17:14:30,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:14:30,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:14:30,531 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c2c2ef6778148209fd5eb7/log.txt.gz 2026-03-24 17:14:30,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:14:31,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:14:31,106 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c2c2ef6778148209fd5eb7/lava-logs.yaml 2026-03-24 17:14:31,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:14:31,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:14:31,278 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c2c2ef6778148209fd5eb7/lava_callback.json.gz 2026-03-24 17:14:31,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:31,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2ef6778148209fd5eb7 HTTP/1.1" 200 None 2026-03-24 17:14:31,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:31,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:14:31,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:31,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2ef6778148209fd5eb7 HTTP/1.1" 200 None 2026-03-24 17:14:31,402 - lava_callback - INFO - Completed processing callback for node 69c2c2ef6778148209fd5eb7 2026-03-24 17:14:34,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:14:34,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:15:37,321 - lava_callback - INFO - Received callback for node 69c023a4c3d73288b0d07f07 INFO: 172.21.0.1:45640 - "POST /node/69c023a4c3d73288b0d07f07 HTTP/1.0" 202 Accepted 2026-03-24 17:15:37,327 - lava_callback - INFO - Starting async job submit for node 69c023a4c3d73288b0d07f07 2026-03-24 17:15:37,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:15:37,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c023a4c3d73288b0d07f07 HTTP/1.1" 200 None 2026-03-24 17:15:37,369 - lava_callback - DEBUG - Processing node 69c023a4c3d73288b0d07f07: baseline-arm64-clabbe 2026-03-24 17:15:37,369 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c023a4c3d73288b0d07f07 2026-03-24 17:15:37,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:15:37,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c023a4c3d73288b0d07f07 HTTP/1.1" 200 None 2026-03-24 17:15:37,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:15:37,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 17:15:37,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:15:37,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c023a4c3d73288b0d07f07 HTTP/1.1" 200 None 2026-03-24 17:15:37,427 - lava_callback - ERROR - Error processing callback for node 69c023a4c3d73288b0d07f07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48996 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:16:37,350 - lava_callback - INFO - Received callback for node 69c023eec3d73288b0d07f11 INFO: 172.21.0.1:49006 - "POST /node/69c023eec3d73288b0d07f11 HTTP/1.0" 202 Accepted 2026-03-24 17:16:37,355 - lava_callback - INFO - Starting async job submit for node 69c023eec3d73288b0d07f11 2026-03-24 17:16:37,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:16:37,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c023eec3d73288b0d07f11 HTTP/1.1" 200 None 2026-03-24 17:16:37,373 - lava_callback - DEBUG - Processing node 69c023eec3d73288b0d07f11: baseline-arm64-clabbe 2026-03-24 17:16:37,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c023eec3d73288b0d07f11 2026-03-24 17:16:37,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:16:37,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c023eec3d73288b0d07f11 HTTP/1.1" 200 None 2026-03-24 17:16:37,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:16:37,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 17:16:37,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:16:37,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c023eec3d73288b0d07f11 HTTP/1.1" 200 None 2026-03-24 17:16:37,434 - lava_callback - ERROR - Error processing callback for node 69c023eec3d73288b0d07f11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49214 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:17:57,396 - lava_callback - INFO - Received callback for node 69c02439c3d73288b0d07f20 INFO: 172.21.0.1:49226 - "POST /node/69c02439c3d73288b0d07f20 HTTP/1.0" 202 Accepted 2026-03-24 17:17:57,402 - lava_callback - INFO - Starting async job submit for node 69c02439c3d73288b0d07f20 2026-03-24 17:17:57,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:17:57,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02439c3d73288b0d07f20 HTTP/1.1" 200 None 2026-03-24 17:17:57,418 - lava_callback - DEBUG - Processing node 69c02439c3d73288b0d07f20: baseline-arm64-clabbe 2026-03-24 17:17:57,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02439c3d73288b0d07f20 2026-03-24 17:17:57,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:17:57,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02439c3d73288b0d07f20 HTTP/1.1" 200 None 2026-03-24 17:17:57,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:17:57,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 17:17:57,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:17:57,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02439c3d73288b0d07f20 HTTP/1.1" 200 None 2026-03-24 17:17:57,479 - lava_callback - ERROR - Error processing callback for node 69c02439c3d73288b0d07f20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36922 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:18:57,422 - lava_callback - INFO - Received callback for node 69c02475c3d73288b0d07f26 INFO: 172.21.0.1:36936 - "POST /node/69c02475c3d73288b0d07f26 HTTP/1.0" 202 Accepted 2026-03-24 17:18:57,427 - lava_callback - INFO - Starting async job submit for node 69c02475c3d73288b0d07f26 2026-03-24 17:18:57,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:18:57,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02475c3d73288b0d07f26 HTTP/1.1" 200 None 2026-03-24 17:18:57,444 - lava_callback - DEBUG - Processing node 69c02475c3d73288b0d07f26: baseline-arm-clabbe 2026-03-24 17:18:57,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02475c3d73288b0d07f26 2026-03-24 17:18:57,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:18:57,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02475c3d73288b0d07f26 HTTP/1.1" 200 None 2026-03-24 17:18:57,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:18:57,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 17:18:57,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:18:57,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02475c3d73288b0d07f26 HTTP/1.1" 200 None 2026-03-24 17:18:57,551 - lava_callback - ERROR - Error processing callback for node 69c02475c3d73288b0d07f26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:19:03,379 - lava_callback - INFO - Received callback for node 69c2c2c26778148209fd5e08 INFO: 172.21.0.1:33736 - "POST /node/69c2c2c26778148209fd5e08 HTTP/1.0" 202 Accepted 2026-03-24 17:19:03,387 - lava_callback - INFO - Starting async job submit for node 69c2c2c26778148209fd5e08 2026-03-24 17:19:03,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:03,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c26778148209fd5e08 HTTP/1.1" 200 None 2026-03-24 17:19:03,449 - lava_callback - DEBUG - Processing node 69c2c2c26778148209fd5e08: baseline-nfs-arm64 2026-03-24 17:19:04,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:19:04,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:19:04,770 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c26778148209fd5e08/log.txt.gz 2026-03-24 17:19:05,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:19:05,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:19:05,659 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c26778148209fd5e08/lava-logs.yaml 2026-03-24 17:19:05,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:19:05,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:19:05,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2c2c26778148209fd5e08/lava_callback.json.gz DEBUG: Setup failed for 69c2c2c26778148209fd5e08. Transitting job node result: incomplete -> fail 2026-03-24 17:19:05,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:05,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2c26778148209fd5e08 HTTP/1.1" 200 None 2026-03-24 17:19:05,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:06,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:19:06,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:06,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2c26778148209fd5e08 HTTP/1.1" 200 None 2026-03-24 17:19:06,158 - lava_callback - INFO - Completed processing callback for node 69c2c2c26778148209fd5e08 INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46198 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:19:34,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:34,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38512 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:19:37,404 - lava_callback - INFO - Received callback for node 69c02499c3d73288b0d07f2f INFO: 172.21.0.1:38522 - "POST /node/69c02499c3d73288b0d07f2f HTTP/1.0" 202 Accepted 2026-03-24 17:19:37,411 - lava_callback - INFO - Starting async job submit for node 69c02499c3d73288b0d07f2f 2026-03-24 17:19:37,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:37,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02499c3d73288b0d07f2f HTTP/1.1" 200 None 2026-03-24 17:19:37,427 - lava_callback - DEBUG - Processing node 69c02499c3d73288b0d07f2f: baseline-arm-clabbe 2026-03-24 17:19:37,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02499c3d73288b0d07f2f 2026-03-24 17:19:37,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:37,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02499c3d73288b0d07f2f HTTP/1.1" 200 None 2026-03-24 17:19:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:37,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 17:19:37,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:19:37,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02499c3d73288b0d07f2f HTTP/1.1" 200 None 2026-03-24 17:19:37,481 - lava_callback - ERROR - Error processing callback for node 69c02499c3d73288b0d07f2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57932 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:20:17,457 - lava_callback - INFO - Received callback for node 69c024bdc3d73288b0d07f3a INFO: 172.21.0.1:57940 - "POST /node/69c024bdc3d73288b0d07f3a HTTP/1.0" 202 Accepted 2026-03-24 17:20:17,463 - lava_callback - INFO - Starting async job submit for node 69c024bdc3d73288b0d07f3a 2026-03-24 17:20:17,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:17,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c024bdc3d73288b0d07f3a HTTP/1.1" 200 None 2026-03-24 17:20:17,480 - lava_callback - DEBUG - Processing node 69c024bdc3d73288b0d07f3a: baseline-arm-clabbe 2026-03-24 17:20:17,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c024bdc3d73288b0d07f3a 2026-03-24 17:20:17,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:17,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c024bdc3d73288b0d07f3a HTTP/1.1" 200 None 2026-03-24 17:20:17,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:17,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 17:20:17,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:17,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c024bdc3d73288b0d07f3a HTTP/1.1" 200 None 2026-03-24 17:20:17,532 - lava_callback - ERROR - Error processing callback for node 69c024bdc3d73288b0d07f3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53112 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:20:37,467 - lava_callback - INFO - Received callback for node 69c024e1c3d73288b0d07f43 INFO: 172.21.0.1:53116 - "POST /node/69c024e1c3d73288b0d07f43 HTTP/1.0" 202 Accepted 2026-03-24 17:20:37,473 - lava_callback - INFO - Starting async job submit for node 69c024e1c3d73288b0d07f43 2026-03-24 17:20:37,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:37,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c024e1c3d73288b0d07f43 HTTP/1.1" 200 None 2026-03-24 17:20:37,494 - lava_callback - DEBUG - Processing node 69c024e1c3d73288b0d07f43: baseline-arm-clabbe 2026-03-24 17:20:37,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c024e1c3d73288b0d07f43 2026-03-24 17:20:37,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:37,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c024e1c3d73288b0d07f43 HTTP/1.1" 200 None 2026-03-24 17:20:37,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:37,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 17:20:37,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:20:37,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c024e1c3d73288b0d07f43 HTTP/1.1" 200 None 2026-03-24 17:20:37,545 - lava_callback - ERROR - Error processing callback for node 69c024e1c3d73288b0d07f43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35036 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:21:17,555 - lava_callback - INFO - Received callback for node 69c02506c3d73288b0d07f50 INFO: 172.21.0.1:35040 - "POST /node/69c02506c3d73288b0d07f50 HTTP/1.0" 202 Accepted 2026-03-24 17:21:17,561 - lava_callback - INFO - Starting async job submit for node 69c02506c3d73288b0d07f50 2026-03-24 17:21:17,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:21:17,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02506c3d73288b0d07f50 HTTP/1.1" 200 None 2026-03-24 17:21:17,576 - lava_callback - DEBUG - Processing node 69c02506c3d73288b0d07f50: baseline-arm-clabbe 2026-03-24 17:21:17,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02506c3d73288b0d07f50 2026-03-24 17:21:17,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:21:17,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02506c3d73288b0d07f50 HTTP/1.1" 200 None 2026-03-24 17:21:17,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:21:17,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 17:21:17,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:21:17,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02506c3d73288b0d07f50 HTTP/1.1" 200 None 2026-03-24 17:21:17,638 - lava_callback - ERROR - Error processing callback for node 69c02506c3d73288b0d07f50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55130 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:23:40,897 - lava_callback - INFO - Received callback for node 69c2c4b96778148209fd6023 INFO: 172.21.0.1:47910 - "POST /node/69c2c4b96778148209fd6023 HTTP/1.0" 202 Accepted 2026-03-24 17:23:40,909 - lava_callback - INFO - Starting async job submit for node 69c2c4b96778148209fd6023 2026-03-24 17:23:41,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:23:41,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c4b96778148209fd6023 HTTP/1.1" 200 None 2026-03-24 17:23:41,034 - lava_callback - DEBUG - Processing node 69c2c4b96778148209fd6023: baseline-arm 2026-03-24 17:23:41,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:23:41,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:23:41,786 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2c4b96778148209fd6023/log.txt.gz 2026-03-24 17:23:42,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:23:42,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:23:42,391 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2c4b96778148209fd6023/lava-logs.yaml 2026-03-24 17:23:42,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:23:42,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:23:42,558 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2c4b96778148209fd6023/lava_callback.json.gz DEBUG: Setup failed for 69c2c4b96778148209fd6023. Transitting job node result: incomplete -> fail 2026-03-24 17:23:42,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:23:42,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c4b96778148209fd6023 HTTP/1.1" 200 None 2026-03-24 17:23:42,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:23:42,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 17:23:42,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:23:42,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c4b96778148209fd6023 HTTP/1.1" 200 None 2026-03-24 17:23:42,842 - lava_callback - INFO - Completed processing callback for node 69c2c4b96778148209fd6023 INFO: 172.21.0.1:47924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:24:04,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:24:04,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44948 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:24:07,562 - lava_callback - INFO - Received callback for node 69c2c4916778148209fd6015 INFO: 172.21.0.1:44964 - "POST /node/69c2c4916778148209fd6015 HTTP/1.0" 202 Accepted 2026-03-24 17:24:07,569 - lava_callback - INFO - Starting async job submit for node 69c2c4916778148209fd6015 2026-03-24 17:24:07,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:24:07,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c4916778148209fd6015 HTTP/1.1" 200 None 2026-03-24 17:24:07,642 - lava_callback - DEBUG - Processing node 69c2c4916778148209fd6015: ltp-smoke 2026-03-24 17:24:08,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:24:08,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:24:08,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2c4916778148209fd6015/log.txt.gz 2026-03-24 17:24:08,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:24:08,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:24:08,794 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2c4916778148209fd6015/lava-logs.yaml 2026-03-24 17:24:08,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:24:08,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:24:08,988 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2c4916778148209fd6015/lava_callback.json.gz 2026-03-24 17:24:09,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:24:09,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c4916778148209fd6015 HTTP/1.1" 200 None 2026-03-24 17:24:09,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:24:09,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9546778148209fd3372 HTTP/1.1" 200 None 2026-03-24 17:24:09,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:24:09,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c4916778148209fd6015 HTTP/1.1" 200 None 2026-03-24 17:24:09,278 - lava_callback - INFO - Completed processing callback for node 69c2c4916778148209fd6015 INFO: 172.21.0.1:46496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60710 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:24:34,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:24:34,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:27:27,994 - lava_callback - INFO - Received callback for node 69c2c81f6778148209fd6118 INFO: 172.21.0.1:47178 - "POST /node/69c2c81f6778148209fd6118 HTTP/1.0" 202 Accepted 2026-03-24 17:27:28,003 - lava_callback - INFO - Starting async job submit for node 69c2c81f6778148209fd6118 2026-03-24 17:27:28,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:28,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c81f6778148209fd6118 HTTP/1.1" 200 None 2026-03-24 17:27:28,088 - lava_callback - DEBUG - Processing node 69c2c81f6778148209fd6118: baseline-arm64-foundriesio 2026-03-24 17:27:29,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:27:29,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:27:29,467 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c81f6778148209fd6118/log.txt.gz 2026-03-24 17:27:30,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:27:30,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:27:30,440 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c81f6778148209fd6118/lava-logs.yaml 2026-03-24 17:27:30,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:27:30,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:27:30,643 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c2c81f6778148209fd6118/lava_callback.json.gz 2026-03-24 17:27:30,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:30,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c81f6778148209fd6118 HTTP/1.1" 200 None 2026-03-24 17:27:30,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:30,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:27:30,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:30,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c81f6778148209fd6118 HTTP/1.1" 200 None 2026-03-24 17:27:30,766 - lava_callback - INFO - Completed processing callback for node 69c2c81f6778148209fd6118 2026-03-24 17:27:34,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:34,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44816 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:27:47,258 - lava_callback - INFO - Received callback for node 69c0268cc3d73288b0d07fc2 INFO: 172.21.0.1:44828 - "POST /node/69c0268cc3d73288b0d07fc2 HTTP/1.0" 202 Accepted 2026-03-24 17:27:47,263 - lava_callback - INFO - Starting async job submit for node 69c0268cc3d73288b0d07fc2 2026-03-24 17:27:47,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:47,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0268cc3d73288b0d07fc2 HTTP/1.1" 200 None 2026-03-24 17:27:47,279 - lava_callback - DEBUG - Processing node 69c0268cc3d73288b0d07fc2: baseline-arm64-baylibre 2026-03-24 17:27:47,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0268cc3d73288b0d07fc2 2026-03-24 17:27:47,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:47,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0268cc3d73288b0d07fc2 HTTP/1.1" 200 None 2026-03-24 17:27:47,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 17:27:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:27:47,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0268cc3d73288b0d07fc2 HTTP/1.1" 200 None 2026-03-24 17:27:47,332 - lava_callback - ERROR - Error processing callback for node 69c0268cc3d73288b0d07fc2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57814 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:29:07,259 - lava_callback - INFO - Received callback for node 69c026d7c3d73288b0d07fcc INFO: 172.21.0.1:57828 - "POST /node/69c026d7c3d73288b0d07fcc HTTP/1.0" 202 Accepted 2026-03-24 17:29:07,264 - lava_callback - INFO - Starting async job submit for node 69c026d7c3d73288b0d07fcc 2026-03-24 17:29:07,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:29:07,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c026d7c3d73288b0d07fcc HTTP/1.1" 200 None 2026-03-24 17:29:07,281 - lava_callback - DEBUG - Processing node 69c026d7c3d73288b0d07fcc: baseline-arm64-baylibre 2026-03-24 17:29:07,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c026d7c3d73288b0d07fcc 2026-03-24 17:29:07,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:29:07,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c026d7c3d73288b0d07fcc HTTP/1.1" 200 None 2026-03-24 17:29:07,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:29:07,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 17:29:07,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:29:07,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c026d7c3d73288b0d07fcc HTTP/1.1" 200 None 2026-03-24 17:29:07,345 - lava_callback - ERROR - Error processing callback for node 69c026d7c3d73288b0d07fcc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60578 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:30:27,342 - lava_callback - INFO - Received callback for node 69c02723c3d73288b0d07fdb INFO: 172.21.0.1:60580 - "POST /node/69c02723c3d73288b0d07fdb HTTP/1.0" 202 Accepted 2026-03-24 17:30:27,348 - lava_callback - INFO - Starting async job submit for node 69c02723c3d73288b0d07fdb 2026-03-24 17:30:27,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:30:27,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02723c3d73288b0d07fdb HTTP/1.1" 200 None 2026-03-24 17:30:27,364 - lava_callback - DEBUG - Processing node 69c02723c3d73288b0d07fdb: baseline-arm64-baylibre 2026-03-24 17:30:27,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02723c3d73288b0d07fdb 2026-03-24 17:30:27,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:30:27,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02723c3d73288b0d07fdb HTTP/1.1" 200 None 2026-03-24 17:30:27,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:30:27,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 17:30:27,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:30:27,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02723c3d73288b0d07fdb HTTP/1.1" 200 None 2026-03-24 17:30:27,418 - lava_callback - ERROR - Error processing callback for node 69c02723c3d73288b0d07fdb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46978 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:31:27,337 - lava_callback - INFO - Received callback for node 69c0276cc3d73288b0d07feb INFO: 172.21.0.1:46984 - "POST /node/69c0276cc3d73288b0d07feb HTTP/1.0" 202 Accepted 2026-03-24 17:31:27,343 - lava_callback - INFO - Starting async job submit for node 69c0276cc3d73288b0d07feb 2026-03-24 17:31:27,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:31:27,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0276cc3d73288b0d07feb HTTP/1.1" 200 None 2026-03-24 17:31:27,359 - lava_callback - DEBUG - Processing node 69c0276cc3d73288b0d07feb: baseline-arm64-baylibre 2026-03-24 17:31:27,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0276cc3d73288b0d07feb 2026-03-24 17:31:27,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:31:27,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0276cc3d73288b0d07feb HTTP/1.1" 200 None 2026-03-24 17:31:27,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:31:27,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:31:27,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:31:27,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0276cc3d73288b0d07feb HTTP/1.1" 200 None 2026-03-24 17:31:27,418 - lava_callback - ERROR - Error processing callback for node 69c0276cc3d73288b0d07feb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:32:47,440 - lava_callback - INFO - Received callback for node 69c027b7c3d73288b0d07ffa INFO: 172.21.0.1:36450 - "POST /node/69c027b7c3d73288b0d07ffa HTTP/1.0" 202 Accepted 2026-03-24 17:32:47,446 - lava_callback - INFO - Starting async job submit for node 69c027b7c3d73288b0d07ffa 2026-03-24 17:32:47,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:47,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c027b7c3d73288b0d07ffa HTTP/1.1" 200 None 2026-03-24 17:32:47,490 - lava_callback - DEBUG - Processing node 69c027b7c3d73288b0d07ffa: baseline-arm64-baylibre 2026-03-24 17:32:47,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c027b7c3d73288b0d07ffa 2026-03-24 17:32:47,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:47,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c027b7c3d73288b0d07ffa HTTP/1.1" 200 None 2026-03-24 17:32:47,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:47,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:32:47,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:47,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c027b7c3d73288b0d07ffa HTTP/1.1" 200 None 2026-03-24 17:32:47,551 - lava_callback - ERROR - Error processing callback for node 69c027b7c3d73288b0d07ffa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:32:49,147 - lava_callback - INFO - Received callback for node 69c2cab26778148209fd61a8 INFO: 172.21.0.1:36464 - "POST /node/69c2cab26778148209fd61a8 HTTP/1.0" 202 Accepted 2026-03-24 17:32:49,153 - lava_callback - INFO - Starting async job submit for node 69c2cab26778148209fd61a8 2026-03-24 17:32:49,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:49,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cab26778148209fd61a8 HTTP/1.1" 200 None 2026-03-24 17:32:49,171 - lava_callback - DEBUG - Processing node 69c2cab26778148209fd61a8: baseline-nfs-arm64 2026-03-24 17:32:49,172 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:32:49,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:32:49,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:32:49,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cab26778148209fd61a8/lava-logs.yaml 2026-03-24 17:32:49,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:32:49,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:32:49,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cab26778148209fd61a8/lava_callback.json.gz 2026-03-24 17:32:49,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:49,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cab26778148209fd61a8 HTTP/1.1" 200 None 2026-03-24 17:32:49,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:49,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:32:49,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:32:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cab26778148209fd61a8 HTTP/1.1" 200 None 2026-03-24 17:32:49,569 - lava_callback - INFO - Completed processing callback for node 69c2cab26778148209fd61a8 INFO: 172.21.0.1:45904 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:33:04,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:33:04,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34650 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:34:07,536 - lava_callback - INFO - Received callback for node 69c02805c3d73288b0d08003 INFO: 172.21.0.1:34656 - "POST /node/69c02805c3d73288b0d08003 HTTP/1.0" 202 Accepted 2026-03-24 17:34:07,544 - lava_callback - INFO - Starting async job submit for node 69c02805c3d73288b0d08003 2026-03-24 17:34:07,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:34:07,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02805c3d73288b0d08003 HTTP/1.1" 200 None 2026-03-24 17:34:07,560 - lava_callback - DEBUG - Processing node 69c02805c3d73288b0d08003: baseline-arm64-baylibre 2026-03-24 17:34:07,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02805c3d73288b0d08003 2026-03-24 17:34:07,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:34:07,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02805c3d73288b0d08003 HTTP/1.1" 200 None 2026-03-24 17:34:07,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:34:07,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:34:07,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:34:07,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02805c3d73288b0d08003 HTTP/1.1" 200 None 2026-03-24 17:34:07,612 - lava_callback - ERROR - Error processing callback for node 69c02805c3d73288b0d08003: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54436 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:35:27,592 - lava_callback - INFO - Received callback for node 69c0284dc3d73288b0d08021 INFO: 172.21.0.1:54446 - "POST /node/69c0284dc3d73288b0d08021 HTTP/1.0" 202 Accepted 2026-03-24 17:35:27,598 - lava_callback - INFO - Starting async job submit for node 69c0284dc3d73288b0d08021 2026-03-24 17:35:27,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:35:27,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284dc3d73288b0d08021 HTTP/1.1" 200 None 2026-03-24 17:35:27,616 - lava_callback - DEBUG - Processing node 69c0284dc3d73288b0d08021: baseline-arm64-baylibre 2026-03-24 17:35:27,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0284dc3d73288b0d08021 2026-03-24 17:35:27,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:35:27,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284dc3d73288b0d08021 HTTP/1.1" 200 None 2026-03-24 17:35:27,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:35:27,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:35:27,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:35:27,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0284dc3d73288b0d08021 HTTP/1.1" 200 None 2026-03-24 17:35:27,676 - lava_callback - ERROR - Error processing callback for node 69c0284dc3d73288b0d08021: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48102 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:36:07,600 - lava_callback - INFO - Received callback for node 69c02872c3d73288b0d080a2 INFO: 172.21.0.1:48104 - "POST /node/69c02872c3d73288b0d080a2 HTTP/1.0" 202 Accepted 2026-03-24 17:36:07,606 - lava_callback - INFO - Starting async job submit for node 69c02872c3d73288b0d080a2 2026-03-24 17:36:07,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:07,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02872c3d73288b0d080a2 HTTP/1.1" 200 None 2026-03-24 17:36:07,622 - lava_callback - DEBUG - Processing node 69c02872c3d73288b0d080a2: baseline-arm64-baylibre 2026-03-24 17:36:07,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02872c3d73288b0d080a2 2026-03-24 17:36:07,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:07,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02872c3d73288b0d080a2 HTTP/1.1" 200 None 2026-03-24 17:36:07,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:07,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:07,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:07,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02872c3d73288b0d080a2 HTTP/1.1" 200 None 2026-03-24 17:36:07,691 - lava_callback - ERROR - Error processing callback for node 69c02872c3d73288b0d080a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:08,281 - lava_callback - INFO - Received callback for node 69c02873c3d73288b0d080a5 INFO: 172.21.0.1:48106 - "POST /node/69c02873c3d73288b0d080a5 HTTP/1.0" 202 Accepted 2026-03-24 17:36:08,287 - lava_callback - INFO - Starting async job submit for node 69c02873c3d73288b0d080a5 2026-03-24 17:36:08,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:08,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02873c3d73288b0d080a5 HTTP/1.1" 200 None 2026-03-24 17:36:08,303 - lava_callback - DEBUG - Processing node 69c02873c3d73288b0d080a5: baseline-arm64-baylibre 2026-03-24 17:36:08,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02873c3d73288b0d080a5 2026-03-24 17:36:08,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:08,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02873c3d73288b0d080a5 HTTP/1.1" 200 None 2026-03-24 17:36:08,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:08,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:08,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:08,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02873c3d73288b0d080a5 HTTP/1.1" 200 None 2026-03-24 17:36:08,358 - lava_callback - ERROR - Error processing callback for node 69c02873c3d73288b0d080a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:08,945 - lava_callback - INFO - Received callback for node 69c02875c3d73288b0d080a8 INFO: 172.21.0.1:48108 - "POST /node/69c02875c3d73288b0d080a8 HTTP/1.0" 202 Accepted 2026-03-24 17:36:08,950 - lava_callback - INFO - Starting async job submit for node 69c02875c3d73288b0d080a8 2026-03-24 17:36:08,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:08,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02875c3d73288b0d080a8 HTTP/1.1" 200 None 2026-03-24 17:36:08,967 - lava_callback - DEBUG - Processing node 69c02875c3d73288b0d080a8: baseline-arm64-baylibre 2026-03-24 17:36:08,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02875c3d73288b0d080a8 2026-03-24 17:36:08,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:08,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02875c3d73288b0d080a8 HTTP/1.1" 200 None 2026-03-24 17:36:08,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:09,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:09,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02875c3d73288b0d080a8 HTTP/1.1" 200 None 2026-03-24 17:36:09,029 - lava_callback - ERROR - Error processing callback for node 69c02875c3d73288b0d080a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:09,639 - lava_callback - INFO - Received callback for node 69c02877c3d73288b0d080af INFO: 172.21.0.1:48122 - "POST /node/69c02877c3d73288b0d080af HTTP/1.0" 202 Accepted 2026-03-24 17:36:09,644 - lava_callback - INFO - Starting async job submit for node 69c02877c3d73288b0d080af 2026-03-24 17:36:09,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:09,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02877c3d73288b0d080af HTTP/1.1" 200 None 2026-03-24 17:36:09,663 - lava_callback - DEBUG - Processing node 69c02877c3d73288b0d080af: baseline-arm64-baylibre 2026-03-24 17:36:09,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02877c3d73288b0d080af 2026-03-24 17:36:09,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:09,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02877c3d73288b0d080af HTTP/1.1" 200 None 2026-03-24 17:36:09,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:09,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:09,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:09,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02877c3d73288b0d080af HTTP/1.1" 200 None 2026-03-24 17:36:09,720 - lava_callback - ERROR - Error processing callback for node 69c02877c3d73288b0d080af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:10,320 - lava_callback - INFO - Received callback for node 69c02879c3d73288b0d080b2 INFO: 172.21.0.1:37740 - "POST /node/69c02879c3d73288b0d080b2 HTTP/1.0" 202 Accepted 2026-03-24 17:36:10,326 - lava_callback - INFO - Starting async job submit for node 69c02879c3d73288b0d080b2 2026-03-24 17:36:10,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:10,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02879c3d73288b0d080b2 HTTP/1.1" 200 None 2026-03-24 17:36:10,351 - lava_callback - DEBUG - Processing node 69c02879c3d73288b0d080b2: baseline-arm64-baylibre 2026-03-24 17:36:10,351 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02879c3d73288b0d080b2 2026-03-24 17:36:10,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:10,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02879c3d73288b0d080b2 HTTP/1.1" 200 None 2026-03-24 17:36:10,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:10,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:10,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:10,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02879c3d73288b0d080b2 HTTP/1.1" 200 None 2026-03-24 17:36:10,420 - lava_callback - ERROR - Error processing callback for node 69c02879c3d73288b0d080b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:10,996 - lava_callback - INFO - Received callback for node 69c0287ac3d73288b0d080b5 INFO: 172.21.0.1:37756 - "POST /node/69c0287ac3d73288b0d080b5 HTTP/1.0" 202 Accepted 2026-03-24 17:36:11,003 - lava_callback - INFO - Starting async job submit for node 69c0287ac3d73288b0d080b5 2026-03-24 17:36:11,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:11,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287ac3d73288b0d080b5 HTTP/1.1" 200 None 2026-03-24 17:36:11,019 - lava_callback - DEBUG - Processing node 69c0287ac3d73288b0d080b5: baseline-arm64-baylibre 2026-03-24 17:36:11,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0287ac3d73288b0d080b5 2026-03-24 17:36:11,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:11,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287ac3d73288b0d080b5 HTTP/1.1" 200 None 2026-03-24 17:36:11,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:11,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:11,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:11,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0287ac3d73288b0d080b5 HTTP/1.1" 200 None 2026-03-24 17:36:11,092 - lava_callback - ERROR - Error processing callback for node 69c0287ac3d73288b0d080b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:36:18,139 - lava_callback - INFO - Received callback for node 69c0288ac3d73288b0d080f6 INFO: 172.21.0.1:37780 - "POST /node/69c0288ac3d73288b0d080f6 HTTP/1.0" 202 Accepted 2026-03-24 17:36:18,145 - lava_callback - INFO - Starting async job submit for node 69c0288ac3d73288b0d080f6 2026-03-24 17:36:18,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288ac3d73288b0d080f6 HTTP/1.1" 200 None 2026-03-24 17:36:18,169 - lava_callback - DEBUG - Processing node 69c0288ac3d73288b0d080f6: baseline-arm64-clabbe 2026-03-24 17:36:18,169 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0288ac3d73288b0d080f6 2026-03-24 17:36:18,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288ac3d73288b0d080f6 HTTP/1.1" 200 None 2026-03-24 17:36:18,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:18,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0288ac3d73288b0d080f6 HTTP/1.1" 200 None 2026-03-24 17:36:18,224 - lava_callback - ERROR - Error processing callback for node 69c0288ac3d73288b0d080f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:18,834 - lava_callback - INFO - Received callback for node 69c0288cc3d73288b0d080fd INFO: 172.21.0.1:37788 - "POST /node/69c0288cc3d73288b0d080fd HTTP/1.0" 202 Accepted 2026-03-24 17:36:18,839 - lava_callback - INFO - Starting async job submit for node 69c0288cc3d73288b0d080fd 2026-03-24 17:36:18,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288cc3d73288b0d080fd HTTP/1.1" 200 None 2026-03-24 17:36:18,855 - lava_callback - DEBUG - Processing node 69c0288cc3d73288b0d080fd: baseline-arm64-clabbe 2026-03-24 17:36:18,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0288cc3d73288b0d080fd 2026-03-24 17:36:18,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288cc3d73288b0d080fd HTTP/1.1" 200 None 2026-03-24 17:36:18,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:18,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:18,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0288cc3d73288b0d080fd HTTP/1.1" 200 None 2026-03-24 17:36:18,921 - lava_callback - ERROR - Error processing callback for node 69c0288cc3d73288b0d080fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:19,522 - lava_callback - INFO - Received callback for node 69c0288dc3d73288b0d08100 INFO: 172.21.0.1:37804 - "POST /node/69c0288dc3d73288b0d08100 HTTP/1.0" 202 Accepted 2026-03-24 17:36:19,528 - lava_callback - INFO - Starting async job submit for node 69c0288dc3d73288b0d08100 2026-03-24 17:36:19,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:19,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288dc3d73288b0d08100 HTTP/1.1" 200 None 2026-03-24 17:36:19,545 - lava_callback - DEBUG - Processing node 69c0288dc3d73288b0d08100: baseline-arm64-clabbe 2026-03-24 17:36:19,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0288dc3d73288b0d08100 2026-03-24 17:36:19,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:19,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288dc3d73288b0d08100 HTTP/1.1" 200 None 2026-03-24 17:36:19,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:19,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:19,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:19,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0288dc3d73288b0d08100 HTTP/1.1" 200 None 2026-03-24 17:36:19,601 - lava_callback - ERROR - Error processing callback for node 69c0288dc3d73288b0d08100: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:20,207 - lava_callback - INFO - Received callback for node 69c0288ec3d73288b0d08107 INFO: 172.21.0.1:58630 - "POST /node/69c0288ec3d73288b0d08107 HTTP/1.0" 202 Accepted 2026-03-24 17:36:20,213 - lava_callback - INFO - Starting async job submit for node 69c0288ec3d73288b0d08107 2026-03-24 17:36:20,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288ec3d73288b0d08107 HTTP/1.1" 200 None 2026-03-24 17:36:20,229 - lava_callback - DEBUG - Processing node 69c0288ec3d73288b0d08107: baseline-arm64-clabbe 2026-03-24 17:36:20,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0288ec3d73288b0d08107 2026-03-24 17:36:20,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288ec3d73288b0d08107 HTTP/1.1" 200 None 2026-03-24 17:36:20,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:20,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0288ec3d73288b0d08107 HTTP/1.1" 200 None 2026-03-24 17:36:20,280 - lava_callback - ERROR - Error processing callback for node 69c0288ec3d73288b0d08107: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 17:36:20,890 - lava_callback - INFO - Received callback for node 69c02890c3d73288b0d0810d INFO: 172.21.0.1:58636 - "POST /node/69c02890c3d73288b0d0810d HTTP/1.0" 202 Accepted 2026-03-24 17:36:20,896 - lava_callback - INFO - Starting async job submit for node 69c02890c3d73288b0d0810d 2026-03-24 17:36:20,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02890c3d73288b0d0810d HTTP/1.1" 200 None 2026-03-24 17:36:20,912 - lava_callback - DEBUG - Processing node 69c02890c3d73288b0d0810d: baseline-arm64-clabbe 2026-03-24 17:36:20,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02890c3d73288b0d0810d 2026-03-24 17:36:20,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02890c3d73288b0d0810d HTTP/1.1" 200 None 2026-03-24 17:36:20,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:20,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:20,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02890c3d73288b0d0810d HTTP/1.1" 200 None 2026-03-24 17:36:20,973 - lava_callback - ERROR - Error processing callback for node 69c02890c3d73288b0d0810d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39660 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:36:38,064 - lava_callback - INFO - Received callback for node 69c02891c3d73288b0d08110 INFO: 172.21.0.1:39664 - "POST /node/69c02891c3d73288b0d08110 HTTP/1.0" 202 Accepted 2026-03-24 17:36:38,070 - lava_callback - INFO - Starting async job submit for node 69c02891c3d73288b0d08110 2026-03-24 17:36:38,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:38,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02891c3d73288b0d08110 HTTP/1.1" 200 None 2026-03-24 17:36:38,086 - lava_callback - DEBUG - Processing node 69c02891c3d73288b0d08110: baseline-arm64-clabbe 2026-03-24 17:36:38,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02891c3d73288b0d08110 2026-03-24 17:36:38,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:38,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02891c3d73288b0d08110 HTTP/1.1" 200 None 2026-03-24 17:36:38,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:38,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 17:36:38,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:38,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02891c3d73288b0d08110 HTTP/1.1" 200 None 2026-03-24 17:36:38,142 - lava_callback - ERROR - Error processing callback for node 69c02891c3d73288b0d08110: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:36:47,652 - lava_callback - INFO - Received callback for node 69c0289cc3d73288b0d0812c INFO: 172.21.0.1:56106 - "POST /node/69c0289cc3d73288b0d0812c HTTP/1.0" 202 Accepted 2026-03-24 17:36:47,658 - lava_callback - INFO - Starting async job submit for node 69c0289cc3d73288b0d0812c 2026-03-24 17:36:47,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:47,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0289cc3d73288b0d0812c HTTP/1.1" 200 None 2026-03-24 17:36:47,674 - lava_callback - DEBUG - Processing node 69c0289cc3d73288b0d0812c: baseline-arm64-baylibre 2026-03-24 17:36:47,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0289cc3d73288b0d0812c 2026-03-24 17:36:47,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:47,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0289cc3d73288b0d0812c HTTP/1.1" 200 None 2026-03-24 17:36:47,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:47,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:36:47,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:36:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0289cc3d73288b0d0812c HTTP/1.1" 200 None 2026-03-24 17:36:47,752 - lava_callback - ERROR - Error processing callback for node 69c0289cc3d73288b0d0812c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39596 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:37:32,491 - lava_callback - INFO - Received callback for node 69c2cba36778148209fd61fc INFO: 172.21.0.1:36848 - "POST /node/69c2cba36778148209fd61fc HTTP/1.0" 202 Accepted 2026-03-24 17:37:32,497 - lava_callback - INFO - Starting async job submit for node 69c2cba36778148209fd61fc 2026-03-24 17:37:32,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:32,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cba36778148209fd61fc HTTP/1.1" 200 None 2026-03-24 17:37:32,542 - lava_callback - DEBUG - Processing node 69c2cba36778148209fd61fc: baseline-arm64 2026-03-24 17:37:32,675 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:37:32,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:37:32,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:37:32,930 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2cba36778148209fd61fc/lava-logs.yaml 2026-03-24 17:37:32,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:37:33,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:37:33,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2cba36778148209fd61fc/lava_callback.json.gz 2026-03-24 17:37:33,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:33,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cba36778148209fd61fc HTTP/1.1" 200 None 2026-03-24 17:37:33,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:33,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:37:33,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:33,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cba36778148209fd61fc HTTP/1.1" 200 None 2026-03-24 17:37:33,231 - lava_callback - INFO - Completed processing callback for node 69c2cba36778148209fd61fc 2026-03-24 17:37:34,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:34,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36994 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:37:47,674 - lava_callback - INFO - Received callback for node 69c028e7c3d73288b0d081e1 INFO: 172.21.0.1:37004 - "POST /node/69c028e7c3d73288b0d081e1 HTTP/1.0" 202 Accepted 2026-03-24 17:37:47,680 - lava_callback - INFO - Starting async job submit for node 69c028e7c3d73288b0d081e1 2026-03-24 17:37:47,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:47,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c028e7c3d73288b0d081e1 HTTP/1.1" 200 None 2026-03-24 17:37:47,696 - lava_callback - DEBUG - Processing node 69c028e7c3d73288b0d081e1: baseline-arm64-baylibre 2026-03-24 17:37:47,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c028e7c3d73288b0d081e1 2026-03-24 17:37:47,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:47,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c028e7c3d73288b0d081e1 HTTP/1.1" 200 None 2026-03-24 17:37:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:47,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:37:47,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:37:47,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c028e7c3d73288b0d081e1 HTTP/1.1" 200 None 2026-03-24 17:37:47,773 - lava_callback - ERROR - Error processing callback for node 69c028e7c3d73288b0d081e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59272 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:39:18,233 - lava_callback - INFO - Received callback for node 69c02937c3d73288b0d08221 INFO: 172.21.0.1:59280 - "POST /node/69c02937c3d73288b0d08221 HTTP/1.0" 202 Accepted 2026-03-24 17:39:18,239 - lava_callback - INFO - Starting async job submit for node 69c02937c3d73288b0d08221 2026-03-24 17:39:18,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:18,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02937c3d73288b0d08221 HTTP/1.1" 200 None 2026-03-24 17:39:18,260 - lava_callback - DEBUG - Processing node 69c02937c3d73288b0d08221: baseline-arm64-clabbe 2026-03-24 17:39:18,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02937c3d73288b0d08221 2026-03-24 17:39:18,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:18,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02937c3d73288b0d08221 HTTP/1.1" 200 None 2026-03-24 17:39:18,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:18,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:39:18,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:18,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02937c3d73288b0d08221 HTTP/1.1" 200 None 2026-03-24 17:39:18,321 - lava_callback - ERROR - Error processing callback for node 69c02937c3d73288b0d08221: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49442 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:39:48,960 - lava_callback - INFO - Received callback for node 69c2c8b36778148209fd612d INFO: 172.21.0.1:49452 - "POST /node/69c2c8b36778148209fd612d HTTP/1.0" 202 Accepted 2026-03-24 17:39:48,970 - lava_callback - INFO - Starting async job submit for node 69c2c8b36778148209fd612d 2026-03-24 17:39:49,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:49,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c8b36778148209fd612d HTTP/1.1" 200 None 2026-03-24 17:39:49,021 - lava_callback - DEBUG - Processing node 69c2c8b36778148209fd612d: baseline-arm64 2026-03-24 17:39:49,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:39:49,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:39:49,576 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c8b36778148209fd612d/log.txt.gz 2026-03-24 17:39:49,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:39:50,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:39:50,011 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c8b36778148209fd612d/lava-logs.yaml 2026-03-24 17:39:50,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:39:50,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:39:50,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c8b36778148209fd612d/lava_callback.json.gz DEBUG: Setup failed for 69c2c8b36778148209fd612d. Transitting job node result: incomplete -> fail 2026-03-24 17:39:50,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:50,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c8b36778148209fd612d HTTP/1.1" 200 None 2026-03-24 17:39:50,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:50,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:39:50,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:39:50,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c8b36778148209fd612d HTTP/1.1" 200 None 2026-03-24 17:39:50,426 - lava_callback - INFO - Completed processing callback for node 69c2c8b36778148209fd612d INFO: 172.21.0.1:59536 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:40:04,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:04,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33776 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:40:38,274 - lava_callback - INFO - Received callback for node 69c02983c3d73288b0d08245 INFO: 172.21.0.1:33780 - "POST /node/69c02983c3d73288b0d08245 HTTP/1.0" 202 Accepted 2026-03-24 17:40:38,279 - lava_callback - INFO - Starting async job submit for node 69c02983c3d73288b0d08245 2026-03-24 17:40:38,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:38,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02983c3d73288b0d08245 HTTP/1.1" 200 None 2026-03-24 17:40:38,299 - lava_callback - DEBUG - Processing node 69c02983c3d73288b0d08245: baseline-arm64-clabbe 2026-03-24 17:40:38,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02983c3d73288b0d08245 2026-03-24 17:40:38,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:38,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02983c3d73288b0d08245 HTTP/1.1" 200 None 2026-03-24 17:40:38,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:38,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:40:38,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:38,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02983c3d73288b0d08245 HTTP/1.1" 200 None 2026-03-24 17:40:38,413 - lava_callback - ERROR - Error processing callback for node 69c02983c3d73288b0d08245: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59546 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:40:57,111 - lava_callback - INFO - Received callback for node 69c2cc466778148209fd6226 INFO: 172.21.0.1:59560 - "POST /node/69c2cc466778148209fd6226 HTTP/1.0" 202 Accepted 2026-03-24 17:40:57,117 - lava_callback - INFO - Starting async job submit for node 69c2cc466778148209fd6226 2026-03-24 17:40:57,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:57,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cc466778148209fd6226 HTTP/1.1" 200 None 2026-03-24 17:40:57,180 - lava_callback - DEBUG - Processing node 69c2cc466778148209fd6226: baseline-arm64 2026-03-24 17:40:57,377 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:40:57,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:40:57,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:40:57,688 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2cc466778148209fd6226/lava-logs.yaml 2026-03-24 17:40:57,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:40:57,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:40:57,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2cc466778148209fd6226/lava_callback.json.gz 2026-03-24 17:40:57,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:57,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cc466778148209fd6226 HTTP/1.1" 200 None 2026-03-24 17:40:57,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:57,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:40:57,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:57,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cc466778148209fd6226 HTTP/1.1" 200 None 2026-03-24 17:40:58,003 - lava_callback - INFO - Completed processing callback for node 69c2cc466778148209fd6226 2026-03-24 17:40:59,632 - lava_callback - INFO - Received callback for node 69c2c2e46778148209fd5e85 INFO: 172.21.0.1:59568 - "POST /node/69c2c2e46778148209fd5e85 HTTP/1.0" 202 Accepted 2026-03-24 17:40:59,638 - lava_callback - INFO - Starting async job submit for node 69c2c2e46778148209fd5e85 2026-03-24 17:40:59,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:40:59,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e46778148209fd5e85 HTTP/1.1" 200 None 2026-03-24 17:40:59,678 - lava_callback - DEBUG - Processing node 69c2c2e46778148209fd5e85: baseline-arm64-broonie 2026-03-24 17:40:59,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:41:00,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:41:00,096 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e46778148209fd5e85/log.txt.gz 2026-03-24 17:41:00,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:41:00,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:41:00,447 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e46778148209fd5e85/lava-logs.yaml 2026-03-24 17:41:00,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:41:00,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:41:00,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e46778148209fd5e85/lava_callback.json.gz 2026-03-24 17:41:00,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:41:00,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e46778148209fd5e85 HTTP/1.1" 200 None 2026-03-24 17:41:00,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:41:00,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:41:00,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:41:00,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2e46778148209fd5e85 HTTP/1.1" 200 None 2026-03-24 17:41:00,692 - lava_callback - INFO - Completed processing callback for node 69c2c2e46778148209fd5e85 2026-03-24 17:41:04,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:41:04,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:44:59,951 - lava_callback - INFO - Received callback for node 69c2c2df6778148209fd5e7b INFO: 172.21.0.1:42712 - "POST /node/69c2c2df6778148209fd5e7b HTTP/1.0" 202 Accepted 2026-03-24 17:44:59,957 - lava_callback - INFO - Starting async job submit for node 69c2c2df6778148209fd5e7b 2026-03-24 17:44:59,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:44:59,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2df6778148209fd5e7b HTTP/1.1" 200 None 2026-03-24 17:44:59,995 - lava_callback - DEBUG - Processing node 69c2c2df6778148209fd5e7b: baseline-arm64-broonie 2026-03-24 17:45:00,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:45:00,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:45:00,402 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2df6778148209fd5e7b/log.txt.gz 2026-03-24 17:45:00,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:45:00,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:45:00,789 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2df6778148209fd5e7b/lava-logs.yaml 2026-03-24 17:45:00,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:45:00,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:45:00,955 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2df6778148209fd5e7b/lava_callback.json.gz 2026-03-24 17:45:00,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:00,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2df6778148209fd5e7b HTTP/1.1" 200 None 2026-03-24 17:45:00,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:01,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:45:01,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:01,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2df6778148209fd5e7b HTTP/1.1" 200 None 2026-03-24 17:45:01,044 - lava_callback - INFO - Completed processing callback for node 69c2c2df6778148209fd5e7b 2026-03-24 17:45:04,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:04,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:45:38,522 - lava_callback - INFO - Received callback for node 69c02aadc3d73288b0d0828c INFO: 172.21.0.1:47318 - "POST /node/69c02aadc3d73288b0d0828c HTTP/1.0" 202 Accepted 2026-03-24 17:45:38,528 - lava_callback - INFO - Starting async job submit for node 69c02aadc3d73288b0d0828c 2026-03-24 17:45:38,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:38,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02aadc3d73288b0d0828c HTTP/1.1" 200 None 2026-03-24 17:45:38,544 - lava_callback - DEBUG - Processing node 69c02aadc3d73288b0d0828c: baseline-arm64-clabbe 2026-03-24 17:45:38,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02aadc3d73288b0d0828c 2026-03-24 17:45:38,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:38,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02aadc3d73288b0d0828c HTTP/1.1" 200 None 2026-03-24 17:45:38,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:38,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:45:38,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:45:38,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02aadc3d73288b0d0828c HTTP/1.1" 200 None 2026-03-24 17:45:38,604 - lava_callback - ERROR - Error processing callback for node 69c02aadc3d73288b0d0828c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33314 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:47:58,538 - lava_callback - INFO - Received callback for node 69c02b41c3d73288b0d082a5 INFO: 172.21.0.1:33326 - "POST /node/69c02b41c3d73288b0d082a5 HTTP/1.0" 202 Accepted 2026-03-24 17:47:58,544 - lava_callback - INFO - Starting async job submit for node 69c02b41c3d73288b0d082a5 2026-03-24 17:47:58,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:47:58,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02b41c3d73288b0d082a5 HTTP/1.1" 200 None 2026-03-24 17:47:58,562 - lava_callback - DEBUG - Processing node 69c02b41c3d73288b0d082a5: baseline-arm64-clabbe 2026-03-24 17:47:58,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02b41c3d73288b0d082a5 2026-03-24 17:47:58,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:47:58,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02b41c3d73288b0d082a5 HTTP/1.1" 200 None 2026-03-24 17:47:58,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:47:58,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:47:58,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:47:58,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02b41c3d73288b0d082a5 HTTP/1.1" 200 None 2026-03-24 17:47:58,645 - lava_callback - ERROR - Error processing callback for node 69c02b41c3d73288b0d082a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46564 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:50:58,851 - lava_callback - INFO - Received callback for node 69c02bf8c3d73288b0d082e6 INFO: 172.21.0.1:46570 - "POST /node/69c02bf8c3d73288b0d082e6 HTTP/1.0" 202 Accepted 2026-03-24 17:50:58,857 - lava_callback - INFO - Starting async job submit for node 69c02bf8c3d73288b0d082e6 2026-03-24 17:50:58,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:50:58,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02bf8c3d73288b0d082e6 HTTP/1.1" 200 None 2026-03-24 17:50:58,872 - lava_callback - DEBUG - Processing node 69c02bf8c3d73288b0d082e6: baseline-arm64-clabbe 2026-03-24 17:50:58,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02bf8c3d73288b0d082e6 2026-03-24 17:50:58,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:50:58,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02bf8c3d73288b0d082e6 HTTP/1.1" 200 None 2026-03-24 17:50:58,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:50:58,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:50:58,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:50:58,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02bf8c3d73288b0d082e6 HTTP/1.1" 200 None 2026-03-24 17:50:58,922 - lava_callback - ERROR - Error processing callback for node 69c02bf8c3d73288b0d082e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51824 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:52:59,101 - lava_callback - INFO - Received callback for node 69c2cf596778148209fd62d5 INFO: 172.21.0.1:51840 - "POST /node/69c2cf596778148209fd62d5 HTTP/1.0" 202 Accepted 2026-03-24 17:52:59,107 - lava_callback - INFO - Starting async job submit for node 69c2cf596778148209fd62d5 2026-03-24 17:52:59,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:52:59,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cf596778148209fd62d5 HTTP/1.1" 200 None 2026-03-24 17:52:59,138 - lava_callback - DEBUG - Processing node 69c2cf596778148209fd62d5: baseline-nfs-arm64 2026-03-24 17:52:59,140 - lava_callback - WARNING - Failed to upload log 2026-03-24 17:52:59,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:52:59,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:52:59,292 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cf596778148209fd62d5/lava-logs.yaml 2026-03-24 17:52:59,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 17:52:59,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 17:52:59,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cf596778148209fd62d5/lava_callback.json.gz 2026-03-24 17:52:59,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:52:59,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cf596778148209fd62d5 HTTP/1.1" 200 None 2026-03-24 17:52:59,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:52:59,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 17:52:59,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:52:59,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cf596778148209fd62d5 HTTP/1.1" 200 None 2026-03-24 17:52:59,484 - lava_callback - INFO - Completed processing callback for node 69c2cf596778148209fd62d5 2026-03-24 17:53:04,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:53:04,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41354 - "GET / HTTP/1.0" 200 OK 2026-03-24 17:53:18,856 - lava_callback - INFO - Received callback for node 69c02c8bc3d73288b0d08307 INFO: 172.21.0.1:41370 - "POST /node/69c02c8bc3d73288b0d08307 HTTP/1.0" 202 Accepted 2026-03-24 17:53:18,862 - lava_callback - INFO - Starting async job submit for node 69c02c8bc3d73288b0d08307 2026-03-24 17:53:18,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:53:18,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02c8bc3d73288b0d08307 HTTP/1.1" 200 None 2026-03-24 17:53:18,877 - lava_callback - DEBUG - Processing node 69c02c8bc3d73288b0d08307: baseline-arm64-clabbe 2026-03-24 17:53:18,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02c8bc3d73288b0d08307 2026-03-24 17:53:18,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:53:18,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02c8bc3d73288b0d08307 HTTP/1.1" 200 None 2026-03-24 17:53:18,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:53:18,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 17:53:18,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 17:53:18,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02c8bc3d73288b0d08307 HTTP/1.1" 200 None 2026-03-24 17:53:18,929 - lava_callback - ERROR - Error processing callback for node 69c02c8bc3d73288b0d08307: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:01:09,078 - lava_callback - INFO - Received callback for node 69c02e5fc3d73288b0d08357 INFO: 172.21.0.1:46680 - "POST /node/69c02e5fc3d73288b0d08357 HTTP/1.0" 202 Accepted 2026-03-24 18:01:09,084 - lava_callback - INFO - Starting async job submit for node 69c02e5fc3d73288b0d08357 2026-03-24 18:01:09,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:09,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02e5fc3d73288b0d08357 HTTP/1.1" 200 None 2026-03-24 18:01:09,100 - lava_callback - DEBUG - Processing node 69c02e5fc3d73288b0d08357: baseline-arm64-baylibre 2026-03-24 18:01:09,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02e5fc3d73288b0d08357 2026-03-24 18:01:09,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:09,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02e5fc3d73288b0d08357 HTTP/1.1" 200 None 2026-03-24 18:01:09,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:09,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:01:09,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:09,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02e5fc3d73288b0d08357 HTTP/1.1" 200 None 2026-03-24 18:01:09,167 - lava_callback - ERROR - Error processing callback for node 69c02e5fc3d73288b0d08357: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 18:01:13,899 - lava_callback - INFO - Received callback for node 69c2cbfb6778148209fd6217 INFO: 172.21.0.1:34856 - "POST /node/69c2cbfb6778148209fd6217 HTTP/1.0" 202 Accepted 2026-03-24 18:01:13,906 - lava_callback - INFO - Starting async job submit for node 69c2cbfb6778148209fd6217 2026-03-24 18:01:13,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:13,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cbfb6778148209fd6217 HTTP/1.1" 200 None 2026-03-24 18:01:13,969 - lava_callback - DEBUG - Processing node 69c2cbfb6778148209fd6217: baseline-arm64 2026-03-24 18:01:14,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:01:14,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:01:14,838 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2cbfb6778148209fd6217/log.txt.gz INFO: 172.21.0.1:34866 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 18:01:15,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:01:15,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:01:15,532 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2cbfb6778148209fd6217/lava-logs.yaml 2026-03-24 18:01:15,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:01:15,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:01:15,723 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2cbfb6778148209fd6217/lava_callback.json.gz DEBUG: Setup failed for 69c2cbfb6778148209fd6217. Transitting job node result: incomplete -> fail 2026-03-24 18:01:15,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:15,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cbfb6778148209fd6217 HTTP/1.1" 200 None 2026-03-24 18:01:15,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:15,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:01:15,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:15,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cbfb6778148209fd6217 HTTP/1.1" 200 None 2026-03-24 18:01:15,979 - lava_callback - INFO - Completed processing callback for node 69c2cbfb6778148209fd6217 INFO: 172.21.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39142 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:01:34,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:34,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51320 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:01:42,130 - lava_callback - INFO - Received callback for node 69c2ccf06778148209fd6249 INFO: 172.21.0.1:36696 - "POST /node/69c2ccf06778148209fd6249 HTTP/1.0" 202 Accepted 2026-03-24 18:01:42,177 - lava_callback - INFO - Starting async job submit for node 69c2ccf06778148209fd6249 2026-03-24 18:01:42,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:42,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ccf06778148209fd6249 HTTP/1.1" 200 None 2026-03-24 18:01:42,217 - lava_callback - DEBUG - Processing node 69c2ccf06778148209fd6249: baseline-arm-pengutronix 2026-03-24 18:01:42,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:01:42,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:01:42,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2ccf06778148209fd6249/log.txt.gz 2026-03-24 18:01:42,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:01:42,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:01:42,752 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2ccf06778148209fd6249/lava-logs.yaml 2026-03-24 18:01:42,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:01:42,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:01:42,889 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2ccf06778148209fd6249/lava_callback.json.gz DEBUG: Setup failed for 69c2ccf06778148209fd6249. Transitting job node result: incomplete -> fail 2026-03-24 18:01:42,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:42,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ccf06778148209fd6249 HTTP/1.1" 200 None 2026-03-24 18:01:42,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:42,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 18:01:42,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:01:42,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ccf06778148209fd6249 HTTP/1.1" 200 None 2026-03-24 18:01:43,089 - lava_callback - INFO - Completed processing callback for node 69c2ccf06778148209fd6249 INFO: 172.21.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49064 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:02:04,488 - lava_callback - INFO - Received callback for node 69c2cca26778148209fd623a INFO: 172.21.0.1:48674 - "POST /node/69c2cca26778148209fd623a HTTP/1.0" 202 Accepted 2026-03-24 18:02:04,496 - lava_callback - INFO - Starting async job submit for node 69c2cca26778148209fd623a 2026-03-24 18:02:04,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:02:04,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cca26778148209fd623a HTTP/1.1" 200 None 2026-03-24 18:02:04,568 - lava_callback - DEBUG - Processing node 69c2cca26778148209fd623a: baseline-nfs-arm64 2026-03-24 18:02:04,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:02:05,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 18:02:05,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:02:05,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:02:05,476 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cca26778148209fd623a/log.txt.gz 2026-03-24 18:02:05,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:02:06,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:02:06,152 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cca26778148209fd623a/lava-logs.yaml 2026-03-24 18:02:06,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:02:06,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:02:06,337 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cca26778148209fd623a/lava_callback.json.gz DEBUG: Setup failed for 69c2cca26778148209fd623a. Transitting job node result: incomplete -> fail 2026-03-24 18:02:06,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:02:06,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cca26778148209fd623a HTTP/1.1" 200 None 2026-03-24 18:02:06,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:02:06,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:02:06,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:02:06,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cca26778148209fd623a HTTP/1.1" 200 None INFO: 172.21.0.1:48690 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:02:06,606 - lava_callback - INFO - Completed processing callback for node 69c2cca26778148209fd623a INFO: 172.21.0.1:56666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56286 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:02:35,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:02:35,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54678 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:04:49,192 - lava_callback - INFO - Received callback for node 69c02f3dc3d73288b0d08372 INFO: 172.21.0.1:54682 - "POST /node/69c02f3dc3d73288b0d08372 HTTP/1.0" 202 Accepted 2026-03-24 18:04:49,197 - lava_callback - INFO - Starting async job submit for node 69c02f3dc3d73288b0d08372 2026-03-24 18:04:49,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:04:49,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02f3dc3d73288b0d08372 HTTP/1.1" 200 None 2026-03-24 18:04:49,213 - lava_callback - DEBUG - Processing node 69c02f3dc3d73288b0d08372: baseline-arm64-baylibre 2026-03-24 18:04:49,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c02f3dc3d73288b0d08372 2026-03-24 18:04:49,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:04:49,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02f3dc3d73288b0d08372 HTTP/1.1" 200 None 2026-03-24 18:04:49,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:04:49,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:04:49,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:04:49,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02f3dc3d73288b0d08372 HTTP/1.1" 200 None 2026-03-24 18:04:49,270 - lava_callback - ERROR - Error processing callback for node 69c02f3dc3d73288b0d08372: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58188 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:05:00,277 - lava_callback - INFO - Received callback for node 69c2c67e6778148209fd60a0 INFO: 172.21.0.1:34912 - "POST /node/69c2c67e6778148209fd60a0 HTTP/1.0" 202 Accepted 2026-03-24 18:05:00,283 - lava_callback - INFO - Starting async job submit for node 69c2c67e6778148209fd60a0 2026-03-24 18:05:00,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:00,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c67e6778148209fd60a0 HTTP/1.1" 200 None 2026-03-24 18:05:00,327 - lava_callback - DEBUG - Processing node 69c2c67e6778148209fd60a0: baseline-arm64 2026-03-24 18:05:00,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:00,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:00,832 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c67e6778148209fd60a0/log.txt.gz 2026-03-24 18:05:00,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:01,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:01,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c67e6778148209fd60a0/lava-logs.yaml 2026-03-24 18:05:01,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:01,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:01,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c67e6778148209fd60a0/lava_callback.json.gz 2026-03-24 18:05:01,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:01,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c67e6778148209fd60a0 HTTP/1.1" 200 None 2026-03-24 18:05:01,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:01,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:05:01,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:01,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c67e6778148209fd60a0 HTTP/1.1" 200 None 2026-03-24 18:05:01,468 - lava_callback - INFO - Completed processing callback for node 69c2c67e6778148209fd60a0 2026-03-24 18:05:05,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:05,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59512 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:05:40,693 - lava_callback - INFO - Received callback for node 69c2ce2e6778148209fd6290 INFO: 172.21.0.1:47196 - "POST /node/69c2ce2e6778148209fd6290 HTTP/1.0" 202 Accepted 2026-03-24 18:05:40,700 - lava_callback - INFO - Starting async job submit for node 69c2ce2e6778148209fd6290 2026-03-24 18:05:40,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:40,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ce2e6778148209fd6290 HTTP/1.1" 200 None 2026-03-24 18:05:40,769 - lava_callback - DEBUG - Processing node 69c2ce2e6778148209fd6290: baseline-arm 2026-03-24 18:05:41,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:41,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:41,508 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2ce2e6778148209fd6290/log.txt.gz 2026-03-24 18:05:41,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:42,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:42,047 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2ce2e6778148209fd6290/lava-logs.yaml 2026-03-24 18:05:42,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:42,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:42,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2ce2e6778148209fd6290/lava_callback.json.gz DEBUG: Setup failed for 69c2ce2e6778148209fd6290. Transitting job node result: incomplete -> fail 2026-03-24 18:05:42,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:42,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ce2e6778148209fd6290 HTTP/1.1" 200 None 2026-03-24 18:05:42,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:42,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 18:05:42,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:42,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ce2e6778148209fd6290 HTTP/1.1" 200 None 2026-03-24 18:05:42,487 - lava_callback - INFO - Completed processing callback for node 69c2ce2e6778148209fd6290 INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:05:49,599 - lava_callback - INFO - Received callback for node 69c2ce576778148209fd629b INFO: 172.21.0.1:47206 - "POST /node/69c2ce576778148209fd629b HTTP/1.0" 202 Accepted 2026-03-24 18:05:49,606 - lava_callback - INFO - Starting async job submit for node 69c2ce576778148209fd629b 2026-03-24 18:05:49,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:49,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ce576778148209fd629b HTTP/1.1" 200 None 2026-03-24 18:05:49,691 - lava_callback - DEBUG - Processing node 69c2ce576778148209fd629b: ltp-smoke 2026-03-24 18:05:50,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:50,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:50,338 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2ce576778148209fd629b/log.txt.gz 2026-03-24 18:05:50,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:50,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:50,822 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2ce576778148209fd629b/lava-logs.yaml 2026-03-24 18:05:50,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:05:50,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:05:50,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2ce576778148209fd629b/lava_callback.json.gz 2026-03-24 18:05:51,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:51,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2ce576778148209fd629b HTTP/1.1" 200 None 2026-03-24 18:05:51,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:51,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9546778148209fd3372 HTTP/1.1" 200 None 2026-03-24 18:05:51,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:05:51,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2ce576778148209fd629b HTTP/1.1" 200 None 2026-03-24 18:05:51,320 - lava_callback - INFO - Completed processing callback for node 69c2ce576778148209fd629b INFO: 172.21.0.1:36344 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:06:05,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:06:05,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34942 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:06:57,785 - lava_callback - INFO - Received callback for node 69c2c6266778148209fd6077 INFO: 172.21.0.1:34958 - "POST /node/69c2c6266778148209fd6077 HTTP/1.0" 202 Accepted 2026-03-24 18:06:57,791 - lava_callback - INFO - Starting async job submit for node 69c2c6266778148209fd6077 2026-03-24 18:06:57,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:06:57,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c6266778148209fd6077 HTTP/1.1" 200 None 2026-03-24 18:06:57,831 - lava_callback - DEBUG - Processing node 69c2c6266778148209fd6077: baseline-arm64 2026-03-24 18:06:58,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:06:58,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:06:58,297 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c6266778148209fd6077/log.txt.gz 2026-03-24 18:06:58,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:06:58,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:06:58,653 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c6266778148209fd6077/lava-logs.yaml 2026-03-24 18:06:58,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:06:58,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:06:58,810 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2c6266778148209fd6077/lava_callback.json.gz 2026-03-24 18:06:58,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:06:58,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c6266778148209fd6077 HTTP/1.1" 200 None 2026-03-24 18:06:58,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:06:58,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:06:58,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:06:58,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c6266778148209fd6077 HTTP/1.1" 200 None 2026-03-24 18:06:58,909 - lava_callback - INFO - Completed processing callback for node 69c2c6266778148209fd6077 2026-03-24 18:07:05,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:07:05,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50936 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:12:09,687 - lava_callback - INFO - Received callback for node 69c030f5c3d73288b0d083b9 INFO: 172.21.0.1:50950 - "POST /node/69c030f5c3d73288b0d083b9 HTTP/1.0" 202 Accepted 2026-03-24 18:12:09,693 - lava_callback - INFO - Starting async job submit for node 69c030f5c3d73288b0d083b9 2026-03-24 18:12:09,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:12:09,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c030f5c3d73288b0d083b9 HTTP/1.1" 200 None 2026-03-24 18:12:09,714 - lava_callback - DEBUG - Processing node 69c030f5c3d73288b0d083b9: baseline-arm64-baylibre 2026-03-24 18:12:09,714 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c030f5c3d73288b0d083b9 2026-03-24 18:12:09,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:12:09,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c030f5c3d73288b0d083b9 HTTP/1.1" 200 None 2026-03-24 18:12:09,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:12:09,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:12:09,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:12:09,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c030f5c3d73288b0d083b9 HTTP/1.1" 200 None 2026-03-24 18:12:09,770 - lava_callback - ERROR - Error processing callback for node 69c030f5c3d73288b0d083b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58504 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:15:30,106 - lava_callback - INFO - Received callback for node 69c2d4886778148209fd63d9 INFO: 172.21.0.1:57456 - "POST /node/69c2d4886778148209fd63d9 HTTP/1.0" 202 Accepted 2026-03-24 18:15:30,113 - lava_callback - INFO - Starting async job submit for node 69c2d4886778148209fd63d9 2026-03-24 18:15:30,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:30,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d4886778148209fd63d9 HTTP/1.1" 200 None 2026-03-24 18:15:30,193 - lava_callback - DEBUG - Processing node 69c2d4886778148209fd63d9: baseline-arm64 2026-03-24 18:15:30,327 - lava_callback - WARNING - Failed to upload log 2026-03-24 18:15:30,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:15:30,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:15:30,584 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d4886778148209fd63d9/lava-logs.yaml 2026-03-24 18:15:30,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:15:30,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:15:30,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d4886778148209fd63d9/lava_callback.json.gz 2026-03-24 18:15:30,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:30,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d4886778148209fd63d9 HTTP/1.1" 200 None 2026-03-24 18:15:30,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:30,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:15:30,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:30,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d4886778148209fd63d9 HTTP/1.1" 200 None 2026-03-24 18:15:30,879 - lava_callback - INFO - Completed processing callback for node 69c2d4886778148209fd63d9 2026-03-24 18:15:35,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:35,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44302 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:15:59,744 - lava_callback - INFO - Received callback for node 69c031d7c3d73288b0d083d9 INFO: 172.21.0.1:41148 - "POST /node/69c031d7c3d73288b0d083d9 HTTP/1.0" 202 Accepted 2026-03-24 18:15:59,749 - lava_callback - INFO - Starting async job submit for node 69c031d7c3d73288b0d083d9 2026-03-24 18:15:59,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:59,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c031d7c3d73288b0d083d9 HTTP/1.1" 200 None 2026-03-24 18:15:59,766 - lava_callback - DEBUG - Processing node 69c031d7c3d73288b0d083d9: baseline-arm64-clabbe 2026-03-24 18:15:59,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c031d7c3d73288b0d083d9 2026-03-24 18:15:59,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:59,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c031d7c3d73288b0d083d9 HTTP/1.1" 200 None 2026-03-24 18:15:59,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:59,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:15:59,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:15:59,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c031d7c3d73288b0d083d9 HTTP/1.1" 200 None 2026-03-24 18:15:59,832 - lava_callback - ERROR - Error processing callback for node 69c031d7c3d73288b0d083d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:17:59,801 - lava_callback - INFO - Received callback for node 69c03246c3d73288b0d083fa INFO: 172.21.0.1:47084 - "POST /node/69c03246c3d73288b0d083fa HTTP/1.0" 202 Accepted 2026-03-24 18:17:59,809 - lava_callback - INFO - Starting async job submit for node 69c03246c3d73288b0d083fa 2026-03-24 18:17:59,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:17:59,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03246c3d73288b0d083fa HTTP/1.1" 200 None 2026-03-24 18:17:59,829 - lava_callback - DEBUG - Processing node 69c03246c3d73288b0d083fa: baseline-arm64-clabbe 2026-03-24 18:17:59,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c03246c3d73288b0d083fa 2026-03-24 18:17:59,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:17:59,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03246c3d73288b0d083fa HTTP/1.1" 200 None 2026-03-24 18:17:59,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:17:59,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:17:59,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:17:59,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03246c3d73288b0d083fa HTTP/1.1" 200 None 2026-03-24 18:17:59,892 - lava_callback - ERROR - Error processing callback for node 69c03246c3d73288b0d083fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52148 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:21:39,919 - lava_callback - INFO - Received callback for node 69c03324c3d73288b0d0841d INFO: 172.21.0.1:43956 - "POST /node/69c03324c3d73288b0d0841d HTTP/1.0" 202 Accepted 2026-03-24 18:21:39,924 - lava_callback - INFO - Starting async job submit for node 69c03324c3d73288b0d0841d 2026-03-24 18:21:39,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:21:39,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03324c3d73288b0d0841d HTTP/1.1" 200 None 2026-03-24 18:21:39,940 - lava_callback - DEBUG - Processing node 69c03324c3d73288b0d0841d: baseline-arm64-clabbe 2026-03-24 18:21:39,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c03324c3d73288b0d0841d 2026-03-24 18:21:39,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:21:39,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03324c3d73288b0d0841d HTTP/1.1" 200 None 2026-03-24 18:21:39,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:21:39,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:21:39,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:21:39,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03324c3d73288b0d0841d HTTP/1.1" 200 None 2026-03-24 18:21:39,998 - lava_callback - ERROR - Error processing callback for node 69c03324c3d73288b0d0841d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:22:31,472 - lava_callback - INFO - Received callback for node 69c2d6316778148209fd640f INFO: 172.21.0.1:36226 - "POST /node/69c2d6316778148209fd640f HTTP/1.0" 202 Accepted 2026-03-24 18:22:31,479 - lava_callback - INFO - Starting async job submit for node 69c2d6316778148209fd640f 2026-03-24 18:22:31,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:22:31,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d6316778148209fd640f HTTP/1.1" 200 None 2026-03-24 18:22:31,525 - lava_callback - DEBUG - Processing node 69c2d6316778148209fd640f: baseline-arm64 2026-03-24 18:22:31,664 - lava_callback - WARNING - Failed to upload log 2026-03-24 18:22:31,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:22:31,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:22:31,947 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d6316778148209fd640f/lava-logs.yaml 2026-03-24 18:22:31,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:22:32,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:22:32,086 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d6316778148209fd640f/lava_callback.json.gz 2026-03-24 18:22:32,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:22:32,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d6316778148209fd640f HTTP/1.1" 200 None 2026-03-24 18:22:32,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:22:32,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:22:32,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:22:32,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d6316778148209fd640f HTTP/1.1" 200 None 2026-03-24 18:22:32,242 - lava_callback - INFO - Completed processing callback for node 69c2d6316778148209fd640f 2026-03-24 18:22:35,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:22:35,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39360 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:23:00,004 - lava_callback - INFO - Received callback for node 69c0336fc3d73288b0d08436 INFO: 172.21.0.1:38318 - "POST /node/69c0336fc3d73288b0d08436 HTTP/1.0" 202 Accepted 2026-03-24 18:23:00,010 - lava_callback - INFO - Starting async job submit for node 69c0336fc3d73288b0d08436 2026-03-24 18:23:00,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:23:00,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0336fc3d73288b0d08436 HTTP/1.1" 200 None 2026-03-24 18:23:00,027 - lava_callback - DEBUG - Processing node 69c0336fc3d73288b0d08436: baseline-arm64-clabbe 2026-03-24 18:23:00,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0336fc3d73288b0d08436 2026-03-24 18:23:00,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:23:00,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0336fc3d73288b0d08436 HTTP/1.1" 200 None 2026-03-24 18:23:00,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:23:00,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:23:00,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:23:00,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0336fc3d73288b0d08436 HTTP/1.1" 200 None 2026-03-24 18:23:00,086 - lava_callback - ERROR - Error processing callback for node 69c0336fc3d73288b0d08436: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60480 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:26:20,353 - lava_callback - INFO - Received callback for node 69c2d58c6778148209fd63fd INFO: 172.21.0.1:35616 - "POST /node/69c2d58c6778148209fd63fd HTTP/1.0" 202 Accepted 2026-03-24 18:26:20,360 - lava_callback - INFO - Starting async job submit for node 69c2d58c6778148209fd63fd 2026-03-24 18:26:20,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:26:20,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d58c6778148209fd63fd HTTP/1.1" 200 None 2026-03-24 18:26:20,413 - lava_callback - DEBUG - Processing node 69c2d58c6778148209fd63fd: baseline-arm64 2026-03-24 18:26:20,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:26:20,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:26:20,957 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d58c6778148209fd63fd/log.txt.gz 2026-03-24 18:26:21,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:26:21,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:26:21,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d58c6778148209fd63fd/lava-logs.yaml 2026-03-24 18:26:21,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:26:21,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:26:21,565 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d58c6778148209fd63fd/lava_callback.json.gz DEBUG: Setup failed for 69c2d58c6778148209fd63fd. Transitting job node result: incomplete -> fail 2026-03-24 18:26:21,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:26:21,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d58c6778148209fd63fd HTTP/1.1" 200 None 2026-03-24 18:26:21,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:26:21,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:26:21,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:26:21,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d58c6778148209fd63fd HTTP/1.1" 200 None 2026-03-24 18:26:21,763 - lava_callback - INFO - Completed processing callback for node 69c2d58c6778148209fd63fd INFO: 172.21.0.1:35628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35634 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:26:35,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:26:35,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44134 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:27:12,310 - lava_callback - INFO - Received callback for node 69c2c2e26778148209fd5e82 INFO: 172.21.0.1:38820 - "POST /node/69c2c2e26778148209fd5e82 HTTP/1.0" 202 Accepted 2026-03-24 18:27:12,316 - lava_callback - INFO - Starting async job submit for node 69c2c2e26778148209fd5e82 2026-03-24 18:27:12,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:12,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e26778148209fd5e82 HTTP/1.1" 200 None 2026-03-24 18:27:12,383 - lava_callback - DEBUG - Processing node 69c2c2e26778148209fd5e82: baseline-arm64-broonie 2026-03-24 18:27:12,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:27:12,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:27:12,734 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e26778148209fd5e82/log.txt.gz 2026-03-24 18:27:12,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:27:13,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:27:13,069 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e26778148209fd5e82/lava-logs.yaml 2026-03-24 18:27:13,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:27:13,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:27:13,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2e26778148209fd5e82/lava_callback.json.gz 2026-03-24 18:27:13,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:13,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2e26778148209fd5e82 HTTP/1.1" 200 None 2026-03-24 18:27:13,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:13,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:27:13,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:13,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2e26778148209fd5e82 HTTP/1.1" 200 None 2026-03-24 18:27:13,356 - lava_callback - INFO - Completed processing callback for node 69c2c2e26778148209fd5e82 INFO: 172.21.0.1:38828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38840 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:27:20,152 - lava_callback - INFO - Received callback for node 69c0346fc3d73288b0d0845b INFO: 172.21.0.1:39536 - "POST /node/69c0346fc3d73288b0d0845b HTTP/1.0" 202 Accepted 2026-03-24 18:27:20,157 - lava_callback - INFO - Starting async job submit for node 69c0346fc3d73288b0d0845b 2026-03-24 18:27:20,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:20,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0346fc3d73288b0d0845b HTTP/1.1" 200 None 2026-03-24 18:27:20,178 - lava_callback - DEBUG - Processing node 69c0346fc3d73288b0d0845b: baseline-arm64-clabbe 2026-03-24 18:27:20,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0346fc3d73288b0d0845b 2026-03-24 18:27:20,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:20,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0346fc3d73288b0d0845b HTTP/1.1" 200 None 2026-03-24 18:27:20,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:20,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:27:20,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:20,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0346fc3d73288b0d0845b HTTP/1.1" 200 None 2026-03-24 18:27:20,245 - lava_callback - ERROR - Error processing callback for node 69c0346fc3d73288b0d0845b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39568 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:27:35,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:27:35,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32910 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:28:20,220 - lava_callback - INFO - Received callback for node 69c034bec3d73288b0d08465 INFO: 172.21.0.1:54114 - "POST /node/69c034bec3d73288b0d08465 HTTP/1.0" 202 Accepted 2026-03-24 18:28:20,226 - lava_callback - INFO - Starting async job submit for node 69c034bec3d73288b0d08465 2026-03-24 18:28:20,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:28:20,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c034bec3d73288b0d08465 HTTP/1.1" 200 None 2026-03-24 18:28:20,242 - lava_callback - DEBUG - Processing node 69c034bec3d73288b0d08465: baseline-arm64-clabbe 2026-03-24 18:28:20,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c034bec3d73288b0d08465 2026-03-24 18:28:20,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:28:20,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c034bec3d73288b0d08465 HTTP/1.1" 200 None 2026-03-24 18:28:20,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:28:20,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-24 18:28:20,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:28:20,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c034bec3d73288b0d08465 HTTP/1.1" 200 None 2026-03-24 18:28:20,298 - lava_callback - ERROR - Error processing callback for node 69c034bec3d73288b0d08465: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:42:29,943 - lava_callback - INFO - Received callback for node 69c2d8d46778148209fd6479 INFO: 172.21.0.1:39034 - "POST /node/69c2d8d46778148209fd6479 HTTP/1.0" 202 Accepted 2026-03-24 18:42:29,951 - lava_callback - INFO - Starting async job submit for node 69c2d8d46778148209fd6479 2026-03-24 18:42:29,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:30,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8d46778148209fd6479 HTTP/1.1" 200 None 2026-03-24 18:42:30,007 - lava_callback - DEBUG - Processing node 69c2d8d46778148209fd6479: baseline-arm64 2026-03-24 18:42:30,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:42:30,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:42:30,539 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d8d46778148209fd6479/log.txt.gz 2026-03-24 18:42:30,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:42:30,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:42:30,984 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d8d46778148209fd6479/lava-logs.yaml 2026-03-24 18:42:30,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:42:31,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:42:31,179 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d8d46778148209fd6479/lava_callback.json.gz DEBUG: Setup failed for 69c2d8d46778148209fd6479. Transitting job node result: incomplete -> fail 2026-03-24 18:42:31,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:31,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8d46778148209fd6479 HTTP/1.1" 200 None 2026-03-24 18:42:31,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:31,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:42:31,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:31,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d8d46778148209fd6479 HTTP/1.1" 200 None 2026-03-24 18:42:31,424 - lava_callback - INFO - Completed processing callback for node 69c2d8d46778148209fd6479 2026-03-24 18:42:35,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:35,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38286 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:42:50,189 - lava_callback - INFO - Received callback for node 69c2d7db6778148209fd6450 INFO: 172.21.0.1:39096 - "POST /node/69c2d7db6778148209fd6450 HTTP/1.0" 202 Accepted 2026-03-24 18:42:50,196 - lava_callback - INFO - Starting async job submit for node 69c2d7db6778148209fd6450 2026-03-24 18:42:50,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:50,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d7db6778148209fd6450 HTTP/1.1" 200 None 2026-03-24 18:42:50,315 - lava_callback - DEBUG - Processing node 69c2d7db6778148209fd6450: baseline-arm64 2026-03-24 18:42:51,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:42:51,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:42:51,184 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d7db6778148209fd6450/log.txt.gz 2026-03-24 18:42:51,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:42:51,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:42:51,860 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d7db6778148209fd6450/lava-logs.yaml 2026-03-24 18:42:51,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:42:52,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:42:52,037 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2d7db6778148209fd6450/lava_callback.json.gz DEBUG: Setup failed for 69c2d7db6778148209fd6450. Transitting job node result: incomplete -> fail 2026-03-24 18:42:52,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:52,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d7db6778148209fd6450 HTTP/1.1" 200 None 2026-03-24 18:42:52,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:52,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:42:52,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:42:52,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d7db6778148209fd6450 HTTP/1.1" 200 None 2026-03-24 18:42:52,292 - lava_callback - INFO - Completed processing callback for node 69c2d7db6778148209fd6450 INFO: 172.21.0.1:39110 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:43:05,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:43:05,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45456 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:45:27,160 - lava_callback - INFO - Received callback for node 69c2d85c6778148209fd6465 INFO: 172.21.0.1:45472 - "POST /node/69c2d85c6778148209fd6465 HTTP/1.0" 202 Accepted 2026-03-24 18:45:27,167 - lava_callback - INFO - Starting async job submit for node 69c2d85c6778148209fd6465 2026-03-24 18:45:27,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:45:27,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d85c6778148209fd6465 HTTP/1.1" 200 None 2026-03-24 18:45:27,252 - lava_callback - DEBUG - Processing node 69c2d85c6778148209fd6465: baseline-nfs-arm64 2026-03-24 18:45:27,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:45:28,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:45:28,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2d85c6778148209fd6465/log.txt.gz 2026-03-24 18:45:28,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:45:28,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:45:28,830 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2d85c6778148209fd6465/lava-logs.yaml 2026-03-24 18:45:28,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:45:29,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:45:29,012 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2d85c6778148209fd6465/lava_callback.json.gz DEBUG: Setup failed for 69c2d85c6778148209fd6465. Transitting job node result: incomplete -> fail 2026-03-24 18:45:29,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:45:29,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d85c6778148209fd6465 HTTP/1.1" 200 None 2026-03-24 18:45:29,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:45:29,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:45:29,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:45:29,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d85c6778148209fd6465 HTTP/1.1" 200 None 2026-03-24 18:45:29,298 - lava_callback - INFO - Completed processing callback for node 69c2d85c6778148209fd6465 2026-03-24 18:45:35,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:45:35,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46930 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:47:41,036 - lava_callback - INFO - Received callback for node 69c03949c3d73288b0d0850e INFO: 172.21.0.1:59542 - "POST /node/69c03949c3d73288b0d0850e HTTP/1.0" 202 Accepted 2026-03-24 18:47:41,041 - lava_callback - INFO - Starting async job submit for node 69c03949c3d73288b0d0850e 2026-03-24 18:47:41,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:47:41,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03949c3d73288b0d0850e HTTP/1.1" 200 None 2026-03-24 18:47:41,057 - lava_callback - DEBUG - Processing node 69c03949c3d73288b0d0850e: baseline-arm-clabbe 2026-03-24 18:47:41,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c03949c3d73288b0d0850e 2026-03-24 18:47:41,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:47:41,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03949c3d73288b0d0850e HTTP/1.1" 200 None 2026-03-24 18:47:41,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:47:41,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 18:47:41,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:47:41,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03949c3d73288b0d0850e HTTP/1.1" 200 None 2026-03-24 18:47:41,126 - lava_callback - ERROR - Error processing callback for node 69c03949c3d73288b0d0850e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:48:20,995 - lava_callback - INFO - Received callback for node 69c0396dc3d73288b0d0851a INFO: 172.21.0.1:37528 - "POST /node/69c0396dc3d73288b0d0851a HTTP/1.0" 202 Accepted 2026-03-24 18:48:21,001 - lava_callback - INFO - Starting async job submit for node 69c0396dc3d73288b0d0851a 2026-03-24 18:48:21,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:48:21,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0396dc3d73288b0d0851a HTTP/1.1" 200 None 2026-03-24 18:48:21,026 - lava_callback - DEBUG - Processing node 69c0396dc3d73288b0d0851a: baseline-arm-clabbe 2026-03-24 18:48:21,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0396dc3d73288b0d0851a 2026-03-24 18:48:21,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:48:21,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0396dc3d73288b0d0851a HTTP/1.1" 200 None 2026-03-24 18:48:21,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:48:21,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 18:48:21,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:48:21,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0396dc3d73288b0d0851a HTTP/1.1" 200 None 2026-03-24 18:48:21,089 - lava_callback - ERROR - Error processing callback for node 69c0396dc3d73288b0d0851a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42216 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:48:54,764 - lava_callback - INFO - Received callback for node 69c2d8a46778148209fd6469 INFO: 172.21.0.1:40046 - "POST /node/69c2d8a46778148209fd6469 HTTP/1.0" 202 Accepted 2026-03-24 18:48:54,771 - lava_callback - INFO - Starting async job submit for node 69c2d8a46778148209fd6469 2026-03-24 18:48:54,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:48:54,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8a46778148209fd6469 HTTP/1.1" 200 None 2026-03-24 18:48:54,839 - lava_callback - DEBUG - Processing node 69c2d8a46778148209fd6469: baseline-arm 2026-03-24 18:48:55,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:48:55,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:48:55,577 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2d8a46778148209fd6469/log.txt.gz 2026-03-24 18:48:55,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:48:56,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:48:56,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2d8a46778148209fd6469/lava-logs.yaml 2026-03-24 18:48:56,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:48:56,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:48:56,346 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2d8a46778148209fd6469/lava_callback.json.gz DEBUG: Setup failed for 69c2d8a46778148209fd6469. Transitting job node result: incomplete -> fail 2026-03-24 18:48:56,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:48:56,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8a46778148209fd6469 HTTP/1.1" 200 None 2026-03-24 18:48:56,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:48:56,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 18:48:56,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:40052 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:48:56,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d8a46778148209fd6469 HTTP/1.1" 200 None 2026-03-24 18:48:56,625 - lava_callback - INFO - Completed processing callback for node 69c2d8a46778148209fd6469 2026-03-24 18:49:01,020 - lava_callback - INFO - Received callback for node 69c03991c3d73288b0d08524 INFO: 172.21.0.1:56502 - "POST /node/69c03991c3d73288b0d08524 HTTP/1.0" 202 Accepted 2026-03-24 18:49:01,026 - lava_callback - INFO - Starting async job submit for node 69c03991c3d73288b0d08524 2026-03-24 18:49:01,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:01,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03991c3d73288b0d08524 HTTP/1.1" 200 None 2026-03-24 18:49:01,042 - lava_callback - DEBUG - Processing node 69c03991c3d73288b0d08524: baseline-arm-clabbe 2026-03-24 18:49:01,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c03991c3d73288b0d08524 2026-03-24 18:49:01,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:01,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03991c3d73288b0d08524 HTTP/1.1" 200 None 2026-03-24 18:49:01,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:01,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 18:49:01,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:01,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03991c3d73288b0d08524 HTTP/1.1" 200 None 2026-03-24 18:49:01,101 - lava_callback - ERROR - Error processing callback for node 69c03991c3d73288b0d08524: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 18:49:05,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:05,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43472 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:49:41,029 - lava_callback - INFO - Received callback for node 69c039c1c3d73288b0d08537 INFO: 172.21.0.1:37814 - "POST /node/69c039c1c3d73288b0d08537 HTTP/1.0" 202 Accepted 2026-03-24 18:49:41,035 - lava_callback - INFO - Starting async job submit for node 69c039c1c3d73288b0d08537 2026-03-24 18:49:41,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:41,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c039c1c3d73288b0d08537 HTTP/1.1" 200 None 2026-03-24 18:49:41,051 - lava_callback - DEBUG - Processing node 69c039c1c3d73288b0d08537: baseline-arm-clabbe 2026-03-24 18:49:41,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c039c1c3d73288b0d08537 2026-03-24 18:49:41,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:41,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c039c1c3d73288b0d08537 HTTP/1.1" 200 None 2026-03-24 18:49:41,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:41,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 18:49:41,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:49:41,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c039c1c3d73288b0d08537 HTTP/1.1" 200 None 2026-03-24 18:49:41,110 - lava_callback - ERROR - Error processing callback for node 69c039c1c3d73288b0d08537: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34660 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:50:10,942 - lava_callback - INFO - Received callback for node 69c2d8cb6778148209fd6473 INFO: 172.21.0.1:38084 - "POST /node/69c2d8cb6778148209fd6473 HTTP/1.0" 202 Accepted 2026-03-24 18:50:10,949 - lava_callback - INFO - Starting async job submit for node 69c2d8cb6778148209fd6473 2026-03-24 18:50:11,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:11,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8cb6778148209fd6473 HTTP/1.1" 200 None 2026-03-24 18:50:11,022 - lava_callback - DEBUG - Processing node 69c2d8cb6778148209fd6473: ltp-smoke 2026-03-24 18:50:11,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:50:11,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:50:11,703 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2d8cb6778148209fd6473/log.txt.gz 2026-03-24 18:50:11,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:50:12,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:50:12,211 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2d8cb6778148209fd6473/lava-logs.yaml 2026-03-24 18:50:12,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:50:12,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:50:12,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2d8cb6778148209fd6473/lava_callback.json.gz 2026-03-24 18:50:12,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:12,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8cb6778148209fd6473 HTTP/1.1" 200 None 2026-03-24 18:50:12,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:12,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9546778148209fd3372 HTTP/1.1" 200 None 2026-03-24 18:50:12,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:12,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d8cb6778148209fd6473 HTTP/1.1" 200 None 2026-03-24 18:50:12,709 - lava_callback - INFO - Completed processing callback for node 69c2d8cb6778148209fd6473 INFO: 172.21.0.1:38094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38106 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:50:21,062 - lava_callback - INFO - Received callback for node 69c039e5c3d73288b0d08542 INFO: 172.21.0.1:51052 - "POST /node/69c039e5c3d73288b0d08542 HTTP/1.0" 202 Accepted 2026-03-24 18:50:21,068 - lava_callback - INFO - Starting async job submit for node 69c039e5c3d73288b0d08542 2026-03-24 18:50:21,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:21,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c039e5c3d73288b0d08542 HTTP/1.1" 200 None 2026-03-24 18:50:21,085 - lava_callback - DEBUG - Processing node 69c039e5c3d73288b0d08542: baseline-arm-clabbe 2026-03-24 18:50:21,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c039e5c3d73288b0d08542 2026-03-24 18:50:21,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:21,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c039e5c3d73288b0d08542 HTTP/1.1" 200 None 2026-03-24 18:50:21,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:21,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 18:50:21,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:21,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c039e5c3d73288b0d08542 HTTP/1.1" 200 None 2026-03-24 18:50:21,155 - lava_callback - ERROR - Error processing callback for node 69c039e5c3d73288b0d08542: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51070 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:50:35,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:35,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35956 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:50:41,999 - lava_callback - INFO - Received callback for node 69c2c2dd6778148209fd5e78 INFO: 172.21.0.1:36084 - "POST /node/69c2c2dd6778148209fd5e78 HTTP/1.0" 202 Accepted 2026-03-24 18:50:42,005 - lava_callback - INFO - Starting async job submit for node 69c2c2dd6778148209fd5e78 2026-03-24 18:50:42,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:42,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2dd6778148209fd5e78 HTTP/1.1" 200 None 2026-03-24 18:50:42,044 - lava_callback - DEBUG - Processing node 69c2c2dd6778148209fd5e78: baseline-arm64-broonie 2026-03-24 18:50:42,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:50:42,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:50:42,464 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2dd6778148209fd5e78/log.txt.gz 2026-03-24 18:50:42,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:50:42,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:50:42,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2dd6778148209fd5e78/lava-logs.yaml 2026-03-24 18:50:42,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:50:43,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:50:43,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2dd6778148209fd5e78/lava_callback.json.gz 2026-03-24 18:50:43,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:43,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2dd6778148209fd5e78 HTTP/1.1" 200 None 2026-03-24 18:50:43,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:43,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:50:43,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:50:43,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2dd6778148209fd5e78 HTTP/1.1" 200 None 2026-03-24 18:50:43,095 - lava_callback - INFO - Completed processing callback for node 69c2c2dd6778148209fd5e78 INFO: 172.21.0.1:36086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:51:05,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:51:05,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK 2026-03-24 18:57:28,351 - lava_callback - INFO - Received callback for node 69c2db336778148209fd64a1 INFO: 172.21.0.1:60564 - "POST /node/69c2db336778148209fd64a1 HTTP/1.0" 202 Accepted 2026-03-24 18:57:28,358 - lava_callback - INFO - Starting async job submit for node 69c2db336778148209fd64a1 2026-03-24 18:57:28,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:57:28,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2db336778148209fd64a1 HTTP/1.1" 200 None 2026-03-24 18:57:28,427 - lava_callback - DEBUG - Processing node 69c2db336778148209fd64a1: baseline-arm64 2026-03-24 18:57:29,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:57:29,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:57:29,342 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2db336778148209fd64a1/log.txt.gz 2026-03-24 18:57:29,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:57:29,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:57:29,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2db336778148209fd64a1/lava-logs.yaml 2026-03-24 18:57:30,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 18:57:30,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 18:57:30,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2db336778148209fd64a1/lava_callback.json.gz DEBUG: Setup failed for 69c2db336778148209fd64a1. Transitting job node result: incomplete -> fail 2026-03-24 18:57:30,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:57:30,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2db336778148209fd64a1 HTTP/1.1" 200 None 2026-03-24 18:57:30,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:57:30,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 18:57:30,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:57:30,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2db336778148209fd64a1 HTTP/1.1" 200 None 2026-03-24 18:57:30,457 - lava_callback - INFO - Completed processing callback for node 69c2db336778148209fd64a1 2026-03-24 18:57:35,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 18:57:35,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46964 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:02:29,170 - lava_callback - INFO - Received callback for node 69c2dc5a6778148209fd64fb INFO: 172.21.0.1:46976 - "POST /node/69c2dc5a6778148209fd64fb HTTP/1.0" 202 Accepted 2026-03-24 19:02:29,177 - lava_callback - INFO - Starting async job submit for node 69c2dc5a6778148209fd64fb 2026-03-24 19:02:29,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:02:29,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dc5a6778148209fd64fb HTTP/1.1" 200 None 2026-03-24 19:02:29,250 - lava_callback - DEBUG - Processing node 69c2dc5a6778148209fd64fb: baseline-nfs-arm64 2026-03-24 19:02:29,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:02:30,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:02:30,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2dc5a6778148209fd64fb/log.txt.gz 2026-03-24 19:02:30,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:02:30,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:02:30,877 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2dc5a6778148209fd64fb/lava-logs.yaml 2026-03-24 19:02:30,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:02:31,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:02:31,047 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2dc5a6778148209fd64fb/lava_callback.json.gz DEBUG: Setup failed for 69c2dc5a6778148209fd64fb. Transitting job node result: incomplete -> fail 2026-03-24 19:02:31,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:02:31,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2dc5a6778148209fd64fb HTTP/1.1" 200 None 2026-03-24 19:02:31,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:02:31,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 19:02:31,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:02:31,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2dc5a6778148209fd64fb HTTP/1.1" 200 None 2026-03-24 19:02:31,341 - lava_callback - INFO - Completed processing callback for node 69c2dc5a6778148209fd64fb 2026-03-24 19:02:35,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:02:35,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46826 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:05:02,747 - lava_callback - INFO - Received callback for node 69c2bebe6778148209fd3437 INFO: 172.21.0.1:33246 - "POST /node/69c2bebe6778148209fd3437 HTTP/1.0" 202 Accepted 2026-03-24 19:05:02,753 - lava_callback - INFO - Starting async job submit for node 69c2bebe6778148209fd3437 2026-03-24 19:05:02,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:05:02,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bebe6778148209fd3437 HTTP/1.1" 200 None 2026-03-24 19:05:02,801 - lava_callback - DEBUG - Processing node 69c2bebe6778148209fd3437: baseline-arm-broonie 2026-03-24 19:05:03,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:05:03,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:05:03,192 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bebe6778148209fd3437/log.txt.gz 2026-03-24 19:05:03,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:05:03,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:05:03,504 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bebe6778148209fd3437/lava-logs.yaml 2026-03-24 19:05:03,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:05:03,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:05:03,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c2bebe6778148209fd3437/lava_callback.json.gz 2026-03-24 19:05:03,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:05:03,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bebe6778148209fd3437 HTTP/1.1" 200 None 2026-03-24 19:05:03,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:05:03,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:05:03,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:05:03,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bebe6778148209fd3437 HTTP/1.1" 200 None 2026-03-24 19:05:03,771 - lava_callback - INFO - Completed processing callback for node 69c2bebe6778148209fd3437 2026-03-24 19:05:05,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:05:05,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37706 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:07:12,231 - lava_callback - INFO - Received callback for node 69c2bee66778148209fd34b6 INFO: 172.21.0.1:35262 - "POST /node/69c2bee66778148209fd34b6 HTTP/1.0" 202 Accepted 2026-03-24 19:07:12,237 - lava_callback - INFO - Starting async job submit for node 69c2bee66778148209fd34b6 2026-03-24 19:07:12,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:12,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bee66778148209fd34b6 HTTP/1.1" 200 None 2026-03-24 19:07:12,288 - lava_callback - DEBUG - Processing node 69c2bee66778148209fd34b6: ltp-cap_bounds 2026-03-24 19:07:12,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:07:12,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:07:12,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c2bee66778148209fd34b6/log.txt.gz 2026-03-24 19:07:12,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:07:13,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:07:13,213 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c2bee66778148209fd34b6/lava-logs.yaml 2026-03-24 19:07:13,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:07:13,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:07:13,369 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c2bee66778148209fd34b6/lava_callback.json.gz 2026-03-24 19:07:13,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:13,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bee66778148209fd34b6 HTTP/1.1" 200 None 2026-03-24 19:07:13,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:13,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:07:13,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:13,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bee66778148209fd34b6 HTTP/1.1" 200 None 2026-03-24 19:07:13,493 - lava_callback - INFO - Completed processing callback for node 69c2bee66778148209fd34b6 INFO: 172.21.0.1:35264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:07:22,938 - lava_callback - INFO - Received callback for node 69c2bee86778148209fd34c8 INFO: 172.21.0.1:36826 - "POST /node/69c2bee86778148209fd34c8 HTTP/1.0" 202 Accepted 2026-03-24 19:07:22,944 - lava_callback - INFO - Starting async job submit for node 69c2bee86778148209fd34c8 2026-03-24 19:07:22,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:22,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bee86778148209fd34c8 HTTP/1.1" 200 None 2026-03-24 19:07:23,000 - lava_callback - DEBUG - Processing node 69c2bee86778148209fd34c8: ltp-containers 2026-03-24 19:07:23,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:07:23,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:07:23,546 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c2bee86778148209fd34c8/log.txt.gz 2026-03-24 19:07:23,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:07:23,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:07:23,978 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c2bee86778148209fd34c8/lava-logs.yaml 2026-03-24 19:07:23,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:07:24,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:07:24,159 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c2bee86778148209fd34c8/lava_callback.json.gz 2026-03-24 19:07:24,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:24,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bee86778148209fd34c8 HTTP/1.1" 200 None 2026-03-24 19:07:24,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:24,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:07:24,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:24,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bee86778148209fd34c8 HTTP/1.1" 200 None 2026-03-24 19:07:24,272 - lava_callback - INFO - Completed processing callback for node 69c2bee86778148209fd34c8 INFO: 172.21.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36844 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:07:35,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:07:35,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39902 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:09:03,058 - lava_callback - INFO - Received callback for node 69c2beea6778148209fd34d3 INFO: 172.21.0.1:53372 - "POST /node/69c2beea6778148209fd34d3 HTTP/1.0" 202 Accepted 2026-03-24 19:09:03,064 - lava_callback - INFO - Starting async job submit for node 69c2beea6778148209fd34d3 2026-03-24 19:09:03,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:09:03,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beea6778148209fd34d3 HTTP/1.1" 200 None 2026-03-24 19:09:03,106 - lava_callback - DEBUG - Processing node 69c2beea6778148209fd34d3: ltp-controllers 2026-03-24 19:09:03,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:09:03,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:09:03,714 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c2beea6778148209fd34d3/log.txt.gz 2026-03-24 19:09:03,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:09:04,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:09:04,259 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c2beea6778148209fd34d3/lava-logs.yaml 2026-03-24 19:09:04,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:09:04,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:09:04,424 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c2beea6778148209fd34d3/lava_callback.json.gz 2026-03-24 19:09:04,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:09:04,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beea6778148209fd34d3 HTTP/1.1" 200 None 2026-03-24 19:09:04,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:09:04,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:09:04,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:09:04,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beea6778148209fd34d3 HTTP/1.1" 200 None 2026-03-24 19:09:04,524 - lava_callback - INFO - Completed processing callback for node 69c2beea6778148209fd34d3 2026-03-24 19:09:05,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:09:05,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:11:23,185 - lava_callback - INFO - Received callback for node 69c2beed6778148209fd34d6 INFO: 172.21.0.1:51852 - "POST /node/69c2beed6778148209fd34d6 HTTP/1.0" 202 Accepted 2026-03-24 19:11:23,191 - lava_callback - INFO - Starting async job submit for node 69c2beed6778148209fd34d6 2026-03-24 19:11:23,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:23,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beed6778148209fd34d6 HTTP/1.1" 200 None 2026-03-24 19:11:23,253 - lava_callback - DEBUG - Processing node 69c2beed6778148209fd34d6: ltp-cve 2026-03-24 19:11:23,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:23,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:23,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c2beed6778148209fd34d6/log.txt.gz 2026-03-24 19:11:24,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:24,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:24,241 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c2beed6778148209fd34d6/lava-logs.yaml 2026-03-24 19:11:24,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:24,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:24,397 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c2beed6778148209fd34d6/lava_callback.json.gz 2026-03-24 19:11:24,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:24,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beed6778148209fd34d6 HTTP/1.1" 200 None 2026-03-24 19:11:24,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:24,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:11:24,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:24,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beed6778148209fd34d6 HTTP/1.1" 200 None 2026-03-24 19:11:24,509 - lava_callback - INFO - Completed processing callback for node 69c2beed6778148209fd34d6 2026-03-24 19:11:24,774 - lava_callback - INFO - Received callback for node 69c2bef36778148209fd34e0 INFO: 172.21.0.1:51860 - "POST /node/69c2bef36778148209fd34e0 HTTP/1.0" 202 Accepted 2026-03-24 19:11:24,784 - lava_callback - INFO - Starting async job submit for node 69c2bef36778148209fd34e0 2026-03-24 19:11:24,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:24,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef36778148209fd34e0 HTTP/1.1" 200 None 2026-03-24 19:11:24,858 - lava_callback - DEBUG - Processing node 69c2bef36778148209fd34e0: ltp-fsx 2026-03-24 19:11:25,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:11:25,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:25,372 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c2bef36778148209fd34e0/log.txt.gz 2026-03-24 19:11:25,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:25,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:25,786 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c2bef36778148209fd34e0/lava-logs.yaml 2026-03-24 19:11:25,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:25,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:25,949 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c2bef36778148209fd34e0/lava_callback.json.gz 2026-03-24 19:11:25,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:25,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef36778148209fd34e0 HTTP/1.1" 200 None 2026-03-24 19:11:25,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:26,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:11:26,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:26,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bef36778148209fd34e0 HTTP/1.1" 200 None 2026-03-24 19:11:26,037 - lava_callback - INFO - Completed processing callback for node 69c2bef36778148209fd34e0 INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:11:31,737 - lava_callback - INFO - Received callback for node 69c2bef16778148209fd34dd INFO: 172.21.0.1:38662 - "POST /node/69c2bef16778148209fd34dd HTTP/1.0" 202 Accepted 2026-03-24 19:11:31,744 - lava_callback - INFO - Starting async job submit for node 69c2bef16778148209fd34dd 2026-03-24 19:11:31,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:31,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef16778148209fd34dd HTTP/1.1" 200 None 2026-03-24 19:11:31,812 - lava_callback - DEBUG - Processing node 69c2bef16778148209fd34dd: ltp-fs_bind 2026-03-24 19:11:32,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:32,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:32,370 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c2bef16778148209fd34dd/log.txt.gz 2026-03-24 19:11:32,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:32,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:32,783 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c2bef16778148209fd34dd/lava-logs.yaml 2026-03-24 19:11:32,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:11:32,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:11:32,967 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c2bef16778148209fd34dd/lava_callback.json.gz 2026-03-24 19:11:32,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:33,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef16778148209fd34dd HTTP/1.1" 200 None 2026-03-24 19:11:33,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:33,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:11:33,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:33,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bef16778148209fd34dd HTTP/1.1" 200 None 2026-03-24 19:11:33,056 - lava_callback - INFO - Completed processing callback for node 69c2bef16778148209fd34dd 2026-03-24 19:11:35,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:11:35,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53802 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:15:03,262 - lava_callback - INFO - Received callback for node 69c2bef46778148209fd34e3 INFO: 172.21.0.1:35510 - "POST /node/69c2bef46778148209fd34e3 HTTP/1.0" 202 Accepted 2026-03-24 19:15:03,268 - lava_callback - INFO - Starting async job submit for node 69c2bef46778148209fd34e3 2026-03-24 19:15:03,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:03,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef46778148209fd34e3 HTTP/1.1" 200 None 2026-03-24 19:15:03,318 - lava_callback - DEBUG - Processing node 69c2bef46778148209fd34e3: ltp-input 2026-03-24 19:15:03,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:03,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:03,799 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c2bef46778148209fd34e3/log.txt.gz 2026-03-24 19:15:04,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:04,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:04,213 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c2bef46778148209fd34e3/lava-logs.yaml 2026-03-24 19:15:04,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:04,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:04,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c2bef46778148209fd34e3/lava_callback.json.gz 2026-03-24 19:15:04,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:04,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef46778148209fd34e3 HTTP/1.1" 200 None 2026-03-24 19:15:04,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:04,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:15:04,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:04,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bef46778148209fd34e3 HTTP/1.1" 200 None 2026-03-24 19:15:04,476 - lava_callback - INFO - Completed processing callback for node 69c2bef46778148209fd34e3 2026-03-24 19:15:05,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:05,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55398 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:15:24,041 - lava_callback - INFO - Received callback for node 69c2bef56778148209fd34e6 INFO: 172.21.0.1:41890 - "POST /node/69c2bef56778148209fd34e6 HTTP/1.0" 202 Accepted 2026-03-24 19:15:24,047 - lava_callback - INFO - Starting async job submit for node 69c2bef56778148209fd34e6 2026-03-24 19:15:24,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:24,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef56778148209fd34e6 HTTP/1.1" 200 None 2026-03-24 19:15:24,088 - lava_callback - DEBUG - Processing node 69c2bef56778148209fd34e6: ltp-ipc 2026-03-24 19:15:24,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:24,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:24,590 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c2bef56778148209fd34e6/log.txt.gz 2026-03-24 19:15:24,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:25,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:25,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c2bef56778148209fd34e6/lava-logs.yaml 2026-03-24 19:15:25,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:25,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:25,205 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c2bef56778148209fd34e6/lava_callback.json.gz 2026-03-24 19:15:25,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:25,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef56778148209fd34e6 HTTP/1.1" 200 None 2026-03-24 19:15:25,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:25,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:15:25,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:25,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bef56778148209fd34e6 HTTP/1.1" 200 None 2026-03-24 19:15:25,286 - lava_callback - INFO - Completed processing callback for node 69c2bef56778148209fd34e6 INFO: 172.21.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41908 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:15:35,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:35,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53266 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:15:51,988 - lava_callback - INFO - Received callback for node 69c2bef96778148209fd34ec INFO: 172.21.0.1:41532 - "POST /node/69c2bef96778148209fd34ec HTTP/1.0" 202 Accepted 2026-03-24 19:15:51,995 - lava_callback - INFO - Starting async job submit for node 69c2bef96778148209fd34ec 2026-03-24 19:15:52,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:52,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef96778148209fd34ec HTTP/1.1" 200 None 2026-03-24 19:15:52,043 - lava_callback - DEBUG - Processing node 69c2bef96778148209fd34ec: ltp-sched 2026-03-24 19:15:52,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:52,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:52,502 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c2bef96778148209fd34ec/log.txt.gz 2026-03-24 19:15:52,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:52,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:52,947 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c2bef96778148209fd34ec/lava-logs.yaml 2026-03-24 19:15:52,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:15:53,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:15:53,120 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c2bef96778148209fd34ec/lava_callback.json.gz 2026-03-24 19:15:53,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:53,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef96778148209fd34ec HTTP/1.1" 200 None 2026-03-24 19:15:53,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:53,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:15:53,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:15:53,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bef96778148209fd34ec HTTP/1.1" 200 None 2026-03-24 19:15:53,267 - lava_callback - INFO - Completed processing callback for node 69c2bef96778148209fd34ec INFO: 172.21.0.1:41538 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:16:03,054 - lava_callback - INFO - Received callback for node 69c2bef76778148209fd34e9 INFO: 172.21.0.1:41940 - "POST /node/69c2bef76778148209fd34e9 HTTP/1.0" 202 Accepted 2026-03-24 19:16:03,060 - lava_callback - INFO - Starting async job submit for node 69c2bef76778148209fd34e9 2026-03-24 19:16:03,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:16:03,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef76778148209fd34e9 HTTP/1.1" 200 None 2026-03-24 19:16:03,100 - lava_callback - DEBUG - Processing node 69c2bef76778148209fd34e9: ltp-pty 2026-03-24 19:16:03,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:16:03,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:16:03,622 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c2bef76778148209fd34e9/log.txt.gz 2026-03-24 19:16:03,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:16:04,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:16:04,041 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c2bef76778148209fd34e9/lava-logs.yaml 2026-03-24 19:16:04,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:16:04,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:16:04,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c2bef76778148209fd34e9/lava_callback.json.gz 2026-03-24 19:16:04,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:16:04,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bef76778148209fd34e9 HTTP/1.1" 200 None 2026-03-24 19:16:04,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:16:04,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:16:04,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:16:04,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bef76778148209fd34e9 HTTP/1.1" 200 None 2026-03-24 19:16:04,294 - lava_callback - INFO - Completed processing callback for node 69c2bef76778148209fd34e9 2026-03-24 19:16:05,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:16:05,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46158 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:19:23,647 - lava_callback - INFO - Received callback for node 69c2befa6778148209fd34f9 INFO: 172.21.0.1:50644 - "POST /node/69c2befa6778148209fd34f9 HTTP/1.0" 202 Accepted 2026-03-24 19:19:23,653 - lava_callback - INFO - Starting async job submit for node 69c2befa6778148209fd34f9 2026-03-24 19:19:23,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:23,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2befa6778148209fd34f9 HTTP/1.1" 200 None 2026-03-24 19:19:23,700 - lava_callback - DEBUG - Processing node 69c2befa6778148209fd34f9: ltp-smoke 2026-03-24 19:19:24,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:19:24,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:19:24,207 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2befa6778148209fd34f9/log.txt.gz 2026-03-24 19:19:24,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:19:24,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:19:24,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2befa6778148209fd34f9/lava-logs.yaml 2026-03-24 19:19:24,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:19:24,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:19:24,803 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c2befa6778148209fd34f9/lava_callback.json.gz 2026-03-24 19:19:24,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:24,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2befa6778148209fd34f9 HTTP/1.1" 200 None 2026-03-24 19:19:24,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:24,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:19:24,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:24,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2befa6778148209fd34f9 HTTP/1.1" 200 None 2026-03-24 19:19:24,890 - lava_callback - INFO - Completed processing callback for node 69c2befa6778148209fd34f9 2026-03-24 19:19:25,233 - lava_callback - INFO - Received callback for node 69c2befd6778148209fd34ff INFO: 172.21.0.1:50656 - "POST /node/69c2befd6778148209fd34ff HTTP/1.0" 202 Accepted 2026-03-24 19:19:25,243 - lava_callback - INFO - Starting async job submit for node 69c2befd6778148209fd34ff 2026-03-24 19:19:25,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:25,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2befd6778148209fd34ff HTTP/1.1" 200 None 2026-03-24 19:19:25,283 - lava_callback - DEBUG - Processing node 69c2befd6778148209fd34ff: ltp-syscalls-ipc INFO: 172.21.0.1:50666 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:19:25,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:19:25,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:19:25,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c2befd6778148209fd34ff/log.txt.gz 2026-03-24 19:19:26,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:19:26,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:19:26,278 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c2befd6778148209fd34ff/lava-logs.yaml 2026-03-24 19:19:26,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:19:26,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:19:26,463 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c2befd6778148209fd34ff/lava_callback.json.gz INFO: 172.21.0.1:50672 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:19:26,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:26,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2befd6778148209fd34ff HTTP/1.1" 200 None 2026-03-24 19:19:26,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:26,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:19:26,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:26,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2befd6778148209fd34ff HTTP/1.1" 200 None 2026-03-24 19:19:26,555 - lava_callback - INFO - Completed processing callback for node 69c2befd6778148209fd34ff 2026-03-24 19:19:35,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:19:35,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52766 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:20:11,648 - lava_callback - INFO - Received callback for node 69c2beff6778148209fd3502 INFO: 172.21.0.1:45216 - "POST /node/69c2beff6778148209fd3502 HTTP/1.0" 202 Accepted 2026-03-24 19:20:11,654 - lava_callback - INFO - Starting async job submit for node 69c2beff6778148209fd3502 2026-03-24 19:20:11,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:11,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beff6778148209fd3502 HTTP/1.1" 200 None 2026-03-24 19:20:11,703 - lava_callback - DEBUG - Processing node 69c2beff6778148209fd3502: ltp-timers 2026-03-24 19:20:12,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:20:12,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:20:12,189 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c2beff6778148209fd3502/log.txt.gz 2026-03-24 19:20:12,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:20:12,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:20:12,589 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c2beff6778148209fd3502/lava-logs.yaml 2026-03-24 19:20:12,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:20:12,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:20:12,776 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c2beff6778148209fd3502/lava_callback.json.gz 2026-03-24 19:20:12,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:12,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2beff6778148209fd3502 HTTP/1.1" 200 None 2026-03-24 19:20:12,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:12,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:20:12,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:12,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2beff6778148209fd3502 HTTP/1.1" 200 None 2026-03-24 19:20:12,910 - lava_callback - INFO - Completed processing callback for node 69c2beff6778148209fd3502 INFO: 172.21.0.1:45220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:20:23,799 - lava_callback - INFO - Received callback for node 69c2bf016778148209fd3505 INFO: 172.21.0.1:46190 - "POST /node/69c2bf016778148209fd3505 HTTP/1.0" 202 Accepted 2026-03-24 19:20:23,806 - lava_callback - INFO - Starting async job submit for node 69c2bf016778148209fd3505 2026-03-24 19:20:23,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:23,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bf016778148209fd3505 HTTP/1.1" 200 None 2026-03-24 19:20:23,869 - lava_callback - DEBUG - Processing node 69c2bf016778148209fd3505: ltp-watchqueue 2026-03-24 19:20:24,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:20:24,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:20:24,389 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c2bf016778148209fd3505/log.txt.gz 2026-03-24 19:20:24,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:20:24,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:20:24,848 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c2bf016778148209fd3505/lava-logs.yaml 2026-03-24 19:20:24,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 19:20:25,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 19:20:25,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c2bf016778148209fd3505/lava_callback.json.gz 2026-03-24 19:20:25,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:25,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bf016778148209fd3505 HTTP/1.1" 200 None 2026-03-24 19:20:25,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:25,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 19:20:25,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:25,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bf016778148209fd3505 HTTP/1.1" 200 None 2026-03-24 19:20:25,141 - lava_callback - INFO - Completed processing callback for node 69c2bf016778148209fd3505 INFO: 172.21.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46212 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:20:35,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:20:35,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:24:53,782 - lava_callback - INFO - Received callback for node 69c04201c3d73288b0d085ff INFO: 172.21.0.1:55360 - "POST /node/69c04201c3d73288b0d085ff HTTP/1.0" 202 Accepted 2026-03-24 19:24:53,788 - lava_callback - INFO - Starting async job submit for node 69c04201c3d73288b0d085ff 2026-03-24 19:24:53,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:24:53,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04201c3d73288b0d085ff HTTP/1.1" 200 None 2026-03-24 19:24:53,804 - lava_callback - DEBUG - Processing node 69c04201c3d73288b0d085ff: baseline-arm64-baylibre 2026-03-24 19:24:53,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c04201c3d73288b0d085ff 2026-03-24 19:24:53,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:24:53,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04201c3d73288b0d085ff HTTP/1.1" 200 None 2026-03-24 19:24:53,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:24:53,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 19:24:53,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:24:53,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c04201c3d73288b0d085ff HTTP/1.1" 200 None 2026-03-24 19:24:53,858 - lava_callback - ERROR - Error processing callback for node 69c04201c3d73288b0d085ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52962 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:26:22,519 - lava_callback - INFO - Received callback for node 69c04258c3d73288b0d0860d INFO: 172.21.0.1:40598 - "POST /node/69c04258c3d73288b0d0860d HTTP/1.0" 202 Accepted 2026-03-24 19:26:22,524 - lava_callback - INFO - Starting async job submit for node 69c04258c3d73288b0d0860d 2026-03-24 19:26:22,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:26:22,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04258c3d73288b0d0860d HTTP/1.1" 200 None 2026-03-24 19:26:22,540 - lava_callback - DEBUG - Processing node 69c04258c3d73288b0d0860d: baseline-arm64-clabbe 2026-03-24 19:26:22,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c04258c3d73288b0d0860d 2026-03-24 19:26:22,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:26:22,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04258c3d73288b0d0860d HTTP/1.1" 200 None 2026-03-24 19:26:22,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:26:22,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 19:26:22,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:26:22,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c04258c3d73288b0d0860d HTTP/1.1" 200 None 2026-03-24 19:26:22,603 - lava_callback - ERROR - Error processing callback for node 69c04258c3d73288b0d0860d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39960 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:27:42,609 - lava_callback - INFO - Received callback for node 69c042a8c3d73288b0d0861e INFO: 172.21.0.1:50652 - "POST /node/69c042a8c3d73288b0d0861e HTTP/1.0" 202 Accepted 2026-03-24 19:27:42,615 - lava_callback - INFO - Starting async job submit for node 69c042a8c3d73288b0d0861e 2026-03-24 19:27:42,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:27:42,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c042a8c3d73288b0d0861e HTTP/1.1" 200 None 2026-03-24 19:27:42,631 - lava_callback - DEBUG - Processing node 69c042a8c3d73288b0d0861e: baseline-arm64-clabbe 2026-03-24 19:27:42,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c042a8c3d73288b0d0861e 2026-03-24 19:27:42,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:27:42,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c042a8c3d73288b0d0861e HTTP/1.1" 200 None 2026-03-24 19:27:42,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:27:42,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 19:27:42,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:27:42,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c042a8c3d73288b0d0861e HTTP/1.1" 200 None 2026-03-24 19:27:42,684 - lava_callback - ERROR - Error processing callback for node 69c042a8c3d73288b0d0861e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32834 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:29:02,676 - lava_callback - INFO - Received callback for node 69c042f7c3d73288b0d0862d INFO: 172.21.0.1:47158 - "POST /node/69c042f7c3d73288b0d0862d HTTP/1.0" 202 Accepted 2026-03-24 19:29:02,681 - lava_callback - INFO - Starting async job submit for node 69c042f7c3d73288b0d0862d 2026-03-24 19:29:02,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:29:02,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c042f7c3d73288b0d0862d HTTP/1.1" 200 None 2026-03-24 19:29:02,697 - lava_callback - DEBUG - Processing node 69c042f7c3d73288b0d0862d: baseline-arm64-clabbe 2026-03-24 19:29:02,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c042f7c3d73288b0d0862d 2026-03-24 19:29:02,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:29:02,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c042f7c3d73288b0d0862d HTTP/1.1" 200 None 2026-03-24 19:29:02,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:29:02,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 19:29:02,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:29:02,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c042f7c3d73288b0d0862d HTTP/1.1" 200 None 2026-03-24 19:29:02,782 - lava_callback - ERROR - Error processing callback for node 69c042f7c3d73288b0d0862d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57320 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:30:22,767 - lava_callback - INFO - Received callback for node 69c04341c3d73288b0d08638 INFO: 172.21.0.1:57020 - "POST /node/69c04341c3d73288b0d08638 HTTP/1.0" 202 Accepted 2026-03-24 19:30:22,773 - lava_callback - INFO - Starting async job submit for node 69c04341c3d73288b0d08638 2026-03-24 19:30:22,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:30:22,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04341c3d73288b0d08638 HTTP/1.1" 200 None 2026-03-24 19:30:22,789 - lava_callback - DEBUG - Processing node 69c04341c3d73288b0d08638: baseline-arm64-clabbe 2026-03-24 19:30:22,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c04341c3d73288b0d08638 2026-03-24 19:30:22,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:30:22,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04341c3d73288b0d08638 HTTP/1.1" 200 None 2026-03-24 19:30:22,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:30:22,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 19:30:22,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:30:22,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c04341c3d73288b0d08638 HTTP/1.1" 200 None 2026-03-24 19:30:22,851 - lava_callback - ERROR - Error processing callback for node 69c04341c3d73288b0d08638: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:31:22,757 - lava_callback - INFO - Received callback for node 69c04388c3d73288b0d08648 INFO: 172.21.0.1:60776 - "POST /node/69c04388c3d73288b0d08648 HTTP/1.0" 202 Accepted 2026-03-24 19:31:22,762 - lava_callback - INFO - Starting async job submit for node 69c04388c3d73288b0d08648 2026-03-24 19:31:22,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:31:22,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04388c3d73288b0d08648 HTTP/1.1" 200 None 2026-03-24 19:31:22,801 - lava_callback - DEBUG - Processing node 69c04388c3d73288b0d08648: baseline-arm64-clabbe 2026-03-24 19:31:22,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c04388c3d73288b0d08648 2026-03-24 19:31:22,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:31:22,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c04388c3d73288b0d08648 HTTP/1.1" 200 None 2026-03-24 19:31:22,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:31:22,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 19:31:22,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:31:22,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c04388c3d73288b0d08648 HTTP/1.1" 200 None 2026-03-24 19:31:22,868 - lava_callback - ERROR - Error processing callback for node 69c04388c3d73288b0d08648: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33834 - "GET / HTTP/1.0" 200 OK 2026-03-24 19:33:02,888 - lava_callback - INFO - Received callback for node 69c043e7c3d73288b0d08654 INFO: 172.21.0.1:34162 - "POST /node/69c043e7c3d73288b0d08654 HTTP/1.0" 202 Accepted 2026-03-24 19:33:02,894 - lava_callback - INFO - Starting async job submit for node 69c043e7c3d73288b0d08654 2026-03-24 19:33:02,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:33:02,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c043e7c3d73288b0d08654 HTTP/1.1" 200 None 2026-03-24 19:33:02,910 - lava_callback - DEBUG - Processing node 69c043e7c3d73288b0d08654: baseline-arm64-clabbe 2026-03-24 19:33:02,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c043e7c3d73288b0d08654 2026-03-24 19:33:02,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:33:02,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c043e7c3d73288b0d08654 HTTP/1.1" 200 None 2026-03-24 19:33:02,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:33:02,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 19:33:02,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 19:33:02,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c043e7c3d73288b0d08654 HTTP/1.1" 200 None 2026-03-24 19:33:02,969 - lava_callback - ERROR - Error processing callback for node 69c043e7c3d73288b0d08654: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58158 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:58174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45834 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60130 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET /sitemap.xml HTTP/1.0" 404 Not Found INFO: 172.21.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49344 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:15:21,140 - lava_callback - INFO - Received callback for node 69c2cd3e6778148209fd625c INFO: 172.21.0.1:50534 - "POST /node/69c2cd3e6778148209fd625c HTTP/1.0" 202 Accepted 2026-03-24 20:15:21,148 - lava_callback - INFO - Starting async job submit for node 69c2cd3e6778148209fd625c 2026-03-24 20:15:21,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:15:21,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cd3e6778148209fd625c HTTP/1.1" 200 None 2026-03-24 20:15:21,206 - lava_callback - DEBUG - Processing node 69c2cd3e6778148209fd625c: baseline-nfs-arm64 2026-03-24 20:15:22,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:15:22,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:15:22,519 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cd3e6778148209fd625c/log.txt.gz 2026-03-24 20:15:23,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:15:23,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:15:23,402 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cd3e6778148209fd625c/lava-logs.yaml 2026-03-24 20:15:23,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:15:23,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:15:23,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2cd3e6778148209fd625c/lava_callback.json.gz DEBUG: Setup failed for 69c2cd3e6778148209fd625c. Transitting job node result: incomplete -> fail 2026-03-24 20:15:23,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:15:23,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2cd3e6778148209fd625c HTTP/1.1" 200 None 2026-03-24 20:15:23,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:15:23,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 20:15:23,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:15:23,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2cd3e6778148209fd625c HTTP/1.1" 200 None 2026-03-24 20:15:23,855 - lava_callback - INFO - Completed processing callback for node 69c2cd3e6778148209fd625c INFO: 172.21.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:15:35,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:15:35,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60838 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:19:54,751 - lava_callback - INFO - Received callback for node 69c2c2d86778148209fd5e6d INFO: 172.21.0.1:60272 - "POST /node/69c2c2d86778148209fd5e6d HTTP/1.0" 202 Accepted 2026-03-24 20:19:54,757 - lava_callback - INFO - Starting async job submit for node 69c2c2d86778148209fd5e6d 2026-03-24 20:19:54,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:19:54,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d86778148209fd5e6d HTTP/1.1" 200 None 2026-03-24 20:19:54,794 - lava_callback - DEBUG - Processing node 69c2c2d86778148209fd5e6d: baseline-arm64-broonie 2026-03-24 20:19:55,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:19:55,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:19:55,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2d86778148209fd5e6d/log.txt.gz 2026-03-24 20:19:55,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:19:55,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:19:55,520 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2d86778148209fd5e6d/lava-logs.yaml 2026-03-24 20:19:55,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:19:55,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:19:55,683 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2d86778148209fd5e6d/lava_callback.json.gz 2026-03-24 20:19:55,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:19:55,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2d86778148209fd5e6d HTTP/1.1" 200 None 2026-03-24 20:19:55,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:19:55,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 20:19:55,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:19:55,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2d86778148209fd5e6d HTTP/1.1" 200 None 2026-03-24 20:19:55,776 - lava_callback - INFO - Completed processing callback for node 69c2c2d86778148209fd5e6d INFO: 172.21.0.1:60284 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:20:05,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:20:05,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:35:48,656 - lava_callback - INFO - Received callback for node 69c2f0e56778148209fd66dd INFO: 172.21.0.1:52760 - "POST /node/69c2f0e56778148209fd66dd HTTP/1.0" 202 Accepted 2026-03-24 20:35:48,666 - lava_callback - INFO - Starting async job submit for node 69c2f0e56778148209fd66dd 2026-03-24 20:35:48,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:35:48,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f0e56778148209fd66dd HTTP/1.1" 200 None 2026-03-24 20:35:48,741 - lava_callback - DEBUG - Processing node 69c2f0e56778148209fd66dd: baseline-nfs-arm64 2026-03-24 20:35:49,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:35:50,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:35:50,023 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2f0e56778148209fd66dd/log.txt.gz 2026-03-24 20:35:50,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:35:50,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:35:50,921 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2f0e56778148209fd66dd/lava-logs.yaml 2026-03-24 20:35:50,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:35:51,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:35:51,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2f0e56778148209fd66dd/lava_callback.json.gz DEBUG: Setup failed for 69c2f0e56778148209fd66dd. Transitting job node result: incomplete -> fail 2026-03-24 20:35:51,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:35:51,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f0e56778148209fd66dd HTTP/1.1" 200 None 2026-03-24 20:35:51,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:35:51,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 20:35:51,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:35:51,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f0e56778148209fd66dd HTTP/1.1" 200 None 2026-03-24 20:35:51,376 - lava_callback - INFO - Completed processing callback for node 69c2f0e56778148209fd66dd INFO: 172.21.0.1:39260 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:36:05,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:36:05,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34090 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:44:58,212 - lava_callback - INFO - Received callback for node 69c054bec3d73288b0d086b8 INFO: 172.21.0.1:34092 - "POST /node/69c054bec3d73288b0d086b8 HTTP/1.0" 202 Accepted 2026-03-24 20:44:58,217 - lava_callback - INFO - Starting async job submit for node 69c054bec3d73288b0d086b8 2026-03-24 20:44:58,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:44:58,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c054bec3d73288b0d086b8 HTTP/1.1" 200 None 2026-03-24 20:44:58,235 - lava_callback - DEBUG - Processing node 69c054bec3d73288b0d086b8: baseline-arm-baylibre 2026-03-24 20:44:58,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c054bec3d73288b0d086b8 2026-03-24 20:44:58,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:44:58,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c054bec3d73288b0d086b8 HTTP/1.1" 200 None 2026-03-24 20:44:58,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:44:58,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:44:58,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:44:58,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c054bec3d73288b0d086b8 HTTP/1.1" 200 None 2026-03-24 20:44:58,294 - lava_callback - ERROR - Error processing callback for node 69c054bec3d73288b0d086b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57130 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:46:06,096 - lava_callback - INFO - Received callback for node 69c054fcc3d73288b0d086c6 INFO: 172.21.0.1:54490 - "POST /node/69c054fcc3d73288b0d086c6 HTTP/1.0" 202 Accepted 2026-03-24 20:46:06,102 - lava_callback - INFO - Starting async job submit for node 69c054fcc3d73288b0d086c6 2026-03-24 20:46:06,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:46:06,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c054fcc3d73288b0d086c6 HTTP/1.1" 200 None 2026-03-24 20:46:06,121 - lava_callback - DEBUG - Processing node 69c054fcc3d73288b0d086c6: baseline-arm-clabbe 2026-03-24 20:46:06,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c054fcc3d73288b0d086c6 2026-03-24 20:46:06,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:46:06,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c054fcc3d73288b0d086c6 HTTP/1.1" 200 None 2026-03-24 20:46:06,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:46:06,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:46:06,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:46:06,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c054fcc3d73288b0d086c6 HTTP/1.1" 200 None 2026-03-24 20:46:06,173 - lava_callback - ERROR - Error processing callback for node 69c054fcc3d73288b0d086c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51762 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:47:06,100 - lava_callback - INFO - Received callback for node 69c05537c3d73288b0d086d5 INFO: 172.21.0.1:57290 - "POST /node/69c05537c3d73288b0d086d5 HTTP/1.0" 202 Accepted 2026-03-24 20:47:06,105 - lava_callback - INFO - Starting async job submit for node 69c05537c3d73288b0d086d5 2026-03-24 20:47:06,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:06,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05537c3d73288b0d086d5 HTTP/1.1" 200 None 2026-03-24 20:47:06,122 - lava_callback - DEBUG - Processing node 69c05537c3d73288b0d086d5: baseline-arm-clabbe 2026-03-24 20:47:06,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05537c3d73288b0d086d5 2026-03-24 20:47:06,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:06,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05537c3d73288b0d086d5 HTTP/1.1" 200 None 2026-03-24 20:47:06,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:06,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:47:06,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:06,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05537c3d73288b0d086d5 HTTP/1.1" 200 None 2026-03-24 20:47:06,175 - lava_callback - ERROR - Error processing callback for node 69c05537c3d73288b0d086d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60156 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:47:26,088 - lava_callback - INFO - Received callback for node 69c0555bc3d73288b0d086e1 INFO: 172.21.0.1:47184 - "POST /node/69c0555bc3d73288b0d086e1 HTTP/1.0" 202 Accepted 2026-03-24 20:47:26,093 - lava_callback - INFO - Starting async job submit for node 69c0555bc3d73288b0d086e1 2026-03-24 20:47:26,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:26,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0555bc3d73288b0d086e1 HTTP/1.1" 200 None 2026-03-24 20:47:26,109 - lava_callback - DEBUG - Processing node 69c0555bc3d73288b0d086e1: baseline-arm-clabbe 2026-03-24 20:47:26,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0555bc3d73288b0d086e1 2026-03-24 20:47:26,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:26,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0555bc3d73288b0d086e1 HTTP/1.1" 200 None 2026-03-24 20:47:26,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:26,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:47:26,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:47:26,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0555bc3d73288b0d086e1 HTTP/1.1" 200 None 2026-03-24 20:47:26,220 - lava_callback - ERROR - Error processing callback for node 69c0555bc3d73288b0d086e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38892 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:49:06,156 - lava_callback - INFO - Received callback for node 69c055b0c3d73288b0d086f0 INFO: 172.21.0.1:45816 - "POST /node/69c055b0c3d73288b0d086f0 HTTP/1.0" 202 Accepted 2026-03-24 20:49:06,162 - lava_callback - INFO - Starting async job submit for node 69c055b0c3d73288b0d086f0 2026-03-24 20:49:06,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:49:06,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c055b0c3d73288b0d086f0 HTTP/1.1" 200 None 2026-03-24 20:49:06,178 - lava_callback - DEBUG - Processing node 69c055b0c3d73288b0d086f0: baseline-arm-clabbe 2026-03-24 20:49:06,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c055b0c3d73288b0d086f0 2026-03-24 20:49:06,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:49:06,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c055b0c3d73288b0d086f0 HTTP/1.1" 200 None 2026-03-24 20:49:06,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:49:06,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:49:06,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:49:06,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c055b0c3d73288b0d086f0 HTTP/1.1" 200 None 2026-03-24 20:49:06,232 - lava_callback - ERROR - Error processing callback for node 69c055b0c3d73288b0d086f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59500 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:50:06,346 - lava_callback - INFO - Received callback for node 69c055ecc3d73288b0d08701 INFO: 172.21.0.1:51852 - "POST /node/69c055ecc3d73288b0d08701 HTTP/1.0" 202 Accepted 2026-03-24 20:50:06,352 - lava_callback - INFO - Starting async job submit for node 69c055ecc3d73288b0d08701 2026-03-24 20:50:06,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:06,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c055ecc3d73288b0d08701 HTTP/1.1" 200 None 2026-03-24 20:50:06,368 - lava_callback - DEBUG - Processing node 69c055ecc3d73288b0d08701: baseline-arm-clabbe 2026-03-24 20:50:06,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c055ecc3d73288b0d08701 2026-03-24 20:50:06,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:06,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c055ecc3d73288b0d08701 HTTP/1.1" 200 None 2026-03-24 20:50:06,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:06,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:50:06,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:06,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c055ecc3d73288b0d08701 HTTP/1.1" 200 None 2026-03-24 20:50:06,433 - lava_callback - ERROR - Error processing callback for node 69c055ecc3d73288b0d08701: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34492 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:50:38,499 - lava_callback - INFO - Received callback for node 69c0560ec3d73288b0d0870c INFO: 172.21.0.1:34504 - "POST /node/69c0560ec3d73288b0d0870c HTTP/1.0" 202 Accepted 2026-03-24 20:50:38,505 - lava_callback - INFO - Starting async job submit for node 69c0560ec3d73288b0d0870c 2026-03-24 20:50:38,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:38,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0560ec3d73288b0d0870c HTTP/1.1" 200 None 2026-03-24 20:50:38,521 - lava_callback - DEBUG - Processing node 69c0560ec3d73288b0d0870c: baseline-arm-baylibre 2026-03-24 20:50:38,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0560ec3d73288b0d0870c 2026-03-24 20:50:38,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:38,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0560ec3d73288b0d0870c HTTP/1.1" 200 None 2026-03-24 20:50:38,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:38,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:50:38,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:50:38,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0560ec3d73288b0d0870c HTTP/1.1" 200 None 2026-03-24 20:50:38,588 - lava_callback - ERROR - Error processing callback for node 69c0560ec3d73288b0d0870c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:51:18,446 - lava_callback - INFO - Received callback for node 69c05632c3d73288b0d08718 INFO: 172.21.0.1:52758 - "POST /node/69c05632c3d73288b0d08718 HTTP/1.0" 202 Accepted 2026-03-24 20:51:18,451 - lava_callback - INFO - Starting async job submit for node 69c05632c3d73288b0d08718 2026-03-24 20:51:18,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:51:18,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05632c3d73288b0d08718 HTTP/1.1" 200 None 2026-03-24 20:51:18,473 - lava_callback - DEBUG - Processing node 69c05632c3d73288b0d08718: baseline-arm-baylibre 2026-03-24 20:51:18,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05632c3d73288b0d08718 2026-03-24 20:51:18,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:51:18,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05632c3d73288b0d08718 HTTP/1.1" 200 None 2026-03-24 20:51:18,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:51:18,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:51:18,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:51:18,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05632c3d73288b0d08718 HTTP/1.1" 200 None 2026-03-24 20:51:18,532 - lava_callback - ERROR - Error processing callback for node 69c05632c3d73288b0d08718: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51568 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:53:06,313 - lava_callback - INFO - Received callback for node 69c056a1c3d73288b0d0872a INFO: 172.21.0.1:33706 - "POST /node/69c056a1c3d73288b0d0872a HTTP/1.0" 202 Accepted 2026-03-24 20:53:06,319 - lava_callback - INFO - Starting async job submit for node 69c056a1c3d73288b0d0872a 2026-03-24 20:53:06,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:53:06,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c056a1c3d73288b0d0872a HTTP/1.1" 200 None 2026-03-24 20:53:06,335 - lava_callback - DEBUG - Processing node 69c056a1c3d73288b0d0872a: baseline-arm-clabbe 2026-03-24 20:53:06,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c056a1c3d73288b0d0872a 2026-03-24 20:53:06,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:53:06,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c056a1c3d73288b0d0872a HTTP/1.1" 200 None 2026-03-24 20:53:06,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:53:06,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:53:06,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:53:06,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c056a1c3d73288b0d0872a HTTP/1.1" 200 None 2026-03-24 20:53:06,391 - lava_callback - ERROR - Error processing callback for node 69c056a1c3d73288b0d0872a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45778 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:56:07,745 - lava_callback - INFO - Received callback for node 69c2f5b16778148209fd66f4 INFO: 172.21.0.1:45790 - "POST /node/69c2f5b16778148209fd66f4 HTTP/1.0" 202 Accepted 2026-03-24 20:56:07,753 - lava_callback - INFO - Starting async job submit for node 69c2f5b16778148209fd66f4 2026-03-24 20:56:07,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:07,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f5b16778148209fd66f4 HTTP/1.1" 200 None 2026-03-24 20:56:07,839 - lava_callback - DEBUG - Processing node 69c2f5b16778148209fd66f4: baseline-nfs-arm64 2026-03-24 20:56:08,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:56:09,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:56:09,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2f5b16778148209fd66f4/log.txt.gz 2026-03-24 20:56:09,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:56:10,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:56:10,063 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2f5b16778148209fd66f4/lava-logs.yaml 2026-03-24 20:56:10,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 20:56:10,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 20:56:10,392 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2f5b16778148209fd66f4/lava_callback.json.gz DEBUG: Setup failed for 69c2f5b16778148209fd66f4. Transitting job node result: incomplete -> fail 2026-03-24 20:56:10,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:10,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f5b16778148209fd66f4 HTTP/1.1" 200 None 2026-03-24 20:56:10,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:10,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 20:56:10,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:10,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f5b16778148209fd66f4 HTTP/1.1" 200 None 2026-03-24 20:56:10,657 - lava_callback - INFO - Completed processing callback for node 69c2f5b16778148209fd66f4 INFO: 172.21.0.1:48352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:56:26,499 - lava_callback - INFO - Received callback for node 69c05767c3d73288b0d08752 INFO: 172.21.0.1:39698 - "POST /node/69c05767c3d73288b0d08752 HTTP/1.0" 202 Accepted 2026-03-24 20:56:26,504 - lava_callback - INFO - Starting async job submit for node 69c05767c3d73288b0d08752 2026-03-24 20:56:26,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39714 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:56:26,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05767c3d73288b0d08752 HTTP/1.1" 200 None 2026-03-24 20:56:26,521 - lava_callback - DEBUG - Processing node 69c05767c3d73288b0d08752: baseline-arm-clabbe 2026-03-24 20:56:26,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05767c3d73288b0d08752 2026-03-24 20:56:26,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:26,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05767c3d73288b0d08752 HTTP/1.1" 200 None 2026-03-24 20:56:26,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:26,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:56:26,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:26,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05767c3d73288b0d08752 HTTP/1.1" 200 None 2026-03-24 20:56:26,614 - lava_callback - ERROR - Error processing callback for node 69c05767c3d73288b0d08752: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39730 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:56:35,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:56:35,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41860 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:58:06,483 - lava_callback - INFO - Received callback for node 69c057dbc3d73288b0d0876d INFO: 172.21.0.1:44072 - "POST /node/69c057dbc3d73288b0d0876d HTTP/1.0" 202 Accepted 2026-03-24 20:58:06,488 - lava_callback - INFO - Starting async job submit for node 69c057dbc3d73288b0d0876d 2026-03-24 20:58:06,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:06,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c057dbc3d73288b0d0876d HTTP/1.1" 200 None 2026-03-24 20:58:06,511 - lava_callback - DEBUG - Processing node 69c057dbc3d73288b0d0876d: baseline-arm-clabbe 2026-03-24 20:58:06,512 - lava_callback - WARNING - No log data found in callback INFO: 172.21.0.1:44086 - "GET / HTTP/1.0" 200 OK Job failure metadata not found for node: 69c057dbc3d73288b0d0876d 2026-03-24 20:58:06,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:06,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c057dbc3d73288b0d0876d HTTP/1.1" 200 None 2026-03-24 20:58:06,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:06,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:58:06,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:06,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c057dbc3d73288b0d0876d HTTP/1.1" 200 None 2026-03-24 20:58:06,563 - lava_callback - ERROR - Error processing callback for node 69c057dbc3d73288b0d0876d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK 2026-03-24 20:58:46,525 - lava_callback - INFO - Received callback for node 69c057ffc3d73288b0d08778 INFO: 172.21.0.1:59008 - "POST /node/69c057ffc3d73288b0d08778 HTTP/1.0" 202 Accepted 2026-03-24 20:58:46,530 - lava_callback - INFO - Starting async job submit for node 69c057ffc3d73288b0d08778 2026-03-24 20:58:46,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:46,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c057ffc3d73288b0d08778 HTTP/1.1" 200 None 2026-03-24 20:58:46,571 - lava_callback - DEBUG - Processing node 69c057ffc3d73288b0d08778: baseline-arm-clabbe 2026-03-24 20:58:46,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c057ffc3d73288b0d08778 2026-03-24 20:58:46,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:46,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c057ffc3d73288b0d08778 HTTP/1.1" 200 None 2026-03-24 20:58:46,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:46,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 20:58:46,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 20:58:46,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c057ffc3d73288b0d08778 HTTP/1.1" 200 None 2026-03-24 20:58:46,662 - lava_callback - ERROR - Error processing callback for node 69c057ffc3d73288b0d08778: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:00:46,655 - lava_callback - INFO - Received callback for node 69c05873c3d73288b0d0878f INFO: 172.21.0.1:46054 - "POST /node/69c05873c3d73288b0d0878f HTTP/1.0" 202 Accepted 2026-03-24 21:00:46,660 - lava_callback - INFO - Starting async job submit for node 69c05873c3d73288b0d0878f 2026-03-24 21:00:46,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:00:46,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05873c3d73288b0d0878f HTTP/1.1" 200 None 2026-03-24 21:00:46,703 - lava_callback - DEBUG - Processing node 69c05873c3d73288b0d0878f: baseline-arm-clabbe 2026-03-24 21:00:46,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05873c3d73288b0d0878f 2026-03-24 21:00:46,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:00:46,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05873c3d73288b0d0878f HTTP/1.1" 200 None 2026-03-24 21:00:46,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:00:46,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 21:00:46,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:00:46,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05873c3d73288b0d0878f HTTP/1.1" 200 None 2026-03-24 21:00:46,760 - lava_callback - ERROR - Error processing callback for node 69c05873c3d73288b0d0878f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55318 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:55334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55830 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:02:46,749 - lava_callback - INFO - Received callback for node 69c058e7c3d73288b0d087a7 INFO: 172.21.0.1:55842 - "POST /node/69c058e7c3d73288b0d087a7 HTTP/1.0" 202 Accepted 2026-03-24 21:02:46,755 - lava_callback - INFO - Starting async job submit for node 69c058e7c3d73288b0d087a7 2026-03-24 21:02:46,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:46,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c058e7c3d73288b0d087a7 HTTP/1.1" 200 None 2026-03-24 21:02:46,778 - lava_callback - DEBUG - Processing node 69c058e7c3d73288b0d087a7: baseline-arm-clabbe 2026-03-24 21:02:46,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c058e7c3d73288b0d087a7 2026-03-24 21:02:46,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:46,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c058e7c3d73288b0d087a7 HTTP/1.1" 200 None 2026-03-24 21:02:46,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:46,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 21:02:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:46,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c058e7c3d73288b0d087a7 HTTP/1.1" 200 None 2026-03-24 21:02:46,836 - lava_callback - ERROR - Error processing callback for node 69c058e7c3d73288b0d087a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 21:02:54,966 - lava_callback - INFO - Received callback for node 69c24d4b4a2c5046eb87ff12 INFO: 172.21.0.1:57276 - "POST /node/69c24d4b4a2c5046eb87ff12 HTTP/1.0" 202 Accepted 2026-03-24 21:02:54,972 - lava_callback - INFO - Starting async job submit for node 69c24d4b4a2c5046eb87ff12 2026-03-24 21:02:54,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:55,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d4b4a2c5046eb87ff12 HTTP/1.1" 200 None 2026-03-24 21:02:55,019 - lava_callback - DEBUG - Processing node 69c24d4b4a2c5046eb87ff12: baseline-x86 2026-03-24 21:02:55,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:02:55,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:02:55,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d4b4a2c5046eb87ff12/log.txt.gz 2026-03-24 21:02:55,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:02:55,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:02:55,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d4b4a2c5046eb87ff12/lava-logs.yaml 2026-03-24 21:02:55,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:02:56,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:02:56,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d4b4a2c5046eb87ff12/lava_callback.json.gz 2026-03-24 21:02:56,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:56,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d4b4a2c5046eb87ff12 HTTP/1.1" 200 None 2026-03-24 21:02:56,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:56,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248fb4a2c5046eb87f8bb HTTP/1.1" 200 None 2026-03-24 21:02:56,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:02:56,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24d4b4a2c5046eb87ff12 HTTP/1.1" 200 None 2026-03-24 21:02:56,115 - lava_callback - INFO - Completed processing callback for node 69c24d4b4a2c5046eb87ff12 INFO: 172.21.0.1:57286 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:03:05,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:05,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39868 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:03:26,682 - lava_callback - INFO - Received callback for node 69c0590bc3d73288b0d087b5 INFO: 172.21.0.1:39876 - "POST /node/69c0590bc3d73288b0d087b5 HTTP/1.0" 202 Accepted 2026-03-24 21:03:26,687 - lava_callback - INFO - Starting async job submit for node 69c0590bc3d73288b0d087b5 2026-03-24 21:03:26,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:26,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0590bc3d73288b0d087b5 HTTP/1.1" 200 None 2026-03-24 21:03:26,718 - lava_callback - DEBUG - Processing node 69c0590bc3d73288b0d087b5: baseline-arm-clabbe 2026-03-24 21:03:26,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0590bc3d73288b0d087b5 2026-03-24 21:03:26,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:26,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0590bc3d73288b0d087b5 HTTP/1.1" 200 None 2026-03-24 21:03:26,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:26,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 21:03:26,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:26,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0590bc3d73288b0d087b5 HTTP/1.1" 200 None 2026-03-24 21:03:26,773 - lava_callback - ERROR - Error processing callback for node 69c0590bc3d73288b0d087b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49816 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:03:59,259 - lava_callback - INFO - Received callback for node 69c05939c3d73288b0d087c4 INFO: 172.21.0.1:49832 - "POST /node/69c05939c3d73288b0d087c4 HTTP/1.0" 202 Accepted 2026-03-24 21:03:59,265 - lava_callback - INFO - Starting async job submit for node 69c05939c3d73288b0d087c4 2026-03-24 21:03:59,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:59,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05939c3d73288b0d087c4 HTTP/1.1" 200 None 2026-03-24 21:03:59,281 - lava_callback - DEBUG - Processing node 69c05939c3d73288b0d087c4: baseline-arm64-baylibre 2026-03-24 21:03:59,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05939c3d73288b0d087c4 2026-03-24 21:03:59,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:59,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05939c3d73288b0d087c4 HTTP/1.1" 200 None 2026-03-24 21:03:59,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:59,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 21:03:59,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:03:59,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05939c3d73288b0d087c4 HTTP/1.1" 200 None 2026-03-24 21:03:59,340 - lava_callback - ERROR - Error processing callback for node 69c05939c3d73288b0d087c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35138 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:05:26,794 - lava_callback - INFO - Received callback for node 69c05990c3d73288b0d087d1 INFO: 172.21.0.1:35154 - "POST /node/69c05990c3d73288b0d087d1 HTTP/1.0" 202 Accepted 2026-03-24 21:05:26,800 - lava_callback - INFO - Starting async job submit for node 69c05990c3d73288b0d087d1 2026-03-24 21:05:26,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:05:26,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05990c3d73288b0d087d1 HTTP/1.1" 200 None 2026-03-24 21:05:26,816 - lava_callback - DEBUG - Processing node 69c05990c3d73288b0d087d1: baseline-arm64-clabbe 2026-03-24 21:05:26,816 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05990c3d73288b0d087d1 2026-03-24 21:05:26,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:05:26,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05990c3d73288b0d087d1 HTTP/1.1" 200 None 2026-03-24 21:05:26,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:05:26,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 21:05:26,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:05:26,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05990c3d73288b0d087d1 HTTP/1.1" 200 None 2026-03-24 21:05:26,886 - lava_callback - ERROR - Error processing callback for node 69c05990c3d73288b0d087d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:06:11,444 - lava_callback - INFO - Received callback for node 69c2a2254a2c5046eb880dfd INFO: 172.21.0.1:55678 - "POST /node/69c2a2254a2c5046eb880dfd HTTP/1.0" 202 Accepted 2026-03-24 21:06:11,450 - lava_callback - INFO - Starting async job submit for node 69c2a2254a2c5046eb880dfd 2026-03-24 21:06:11,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:11,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a2254a2c5046eb880dfd HTTP/1.1" 200 None 2026-03-24 21:06:11,483 - lava_callback - DEBUG - Processing node 69c2a2254a2c5046eb880dfd: baseline-x86 2026-03-24 21:06:11,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:06:11,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:06:11,933 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c2a2254a2c5046eb880dfd/log.txt.gz 2026-03-24 21:06:12,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:06:12,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:06:12,313 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c2a2254a2c5046eb880dfd/lava-logs.yaml 2026-03-24 21:06:12,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:06:12,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:06:12,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c2a2254a2c5046eb880dfd/lava_callback.json.gz 2026-03-24 21:06:12,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:12,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2a2254a2c5046eb880dfd HTTP/1.1" 200 None 2026-03-24 21:06:12,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:12,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248fb4a2c5046eb87f8bb HTTP/1.1" 200 None 2026-03-24 21:06:12,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:12,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2a2254a2c5046eb880dfd HTTP/1.1" 200 None 2026-03-24 21:06:12,540 - lava_callback - INFO - Completed processing callback for node 69c2a2254a2c5046eb880dfd INFO: 172.21.0.1:55688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34776 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:06:35,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:35,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53486 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:06:46,920 - lava_callback - INFO - Received callback for node 69c059e0c3d73288b0d087da INFO: 172.21.0.1:53502 - "POST /node/69c059e0c3d73288b0d087da HTTP/1.0" 202 Accepted 2026-03-24 21:06:46,926 - lava_callback - INFO - Starting async job submit for node 69c059e0c3d73288b0d087da 2026-03-24 21:06:46,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:46,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c059e0c3d73288b0d087da HTTP/1.1" 200 None 2026-03-24 21:06:46,944 - lava_callback - DEBUG - Processing node 69c059e0c3d73288b0d087da: baseline-arm64-clabbe 2026-03-24 21:06:46,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c059e0c3d73288b0d087da 2026-03-24 21:06:46,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:46,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c059e0c3d73288b0d087da HTTP/1.1" 200 None 2026-03-24 21:06:46,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:46,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 21:06:46,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:06:47,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c059e0c3d73288b0d087da HTTP/1.1" 200 None 2026-03-24 21:06:47,013 - lava_callback - ERROR - Error processing callback for node 69c059e0c3d73288b0d087da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40762 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:08:06,932 - lava_callback - INFO - Received callback for node 69c05a32c3d73288b0d087e6 INFO: 172.21.0.1:40772 - "POST /node/69c05a32c3d73288b0d087e6 HTTP/1.0" 202 Accepted 2026-03-24 21:08:06,937 - lava_callback - INFO - Starting async job submit for node 69c05a32c3d73288b0d087e6 2026-03-24 21:08:06,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:08:06,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05a32c3d73288b0d087e6 HTTP/1.1" 200 None 2026-03-24 21:08:06,954 - lava_callback - DEBUG - Processing node 69c05a32c3d73288b0d087e6: baseline-arm64-clabbe 2026-03-24 21:08:06,954 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05a32c3d73288b0d087e6 2026-03-24 21:08:06,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:08:06,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05a32c3d73288b0d087e6 HTTP/1.1" 200 None 2026-03-24 21:08:06,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:08:06,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 21:08:06,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:08:07,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05a32c3d73288b0d087e6 HTTP/1.1" 200 None 2026-03-24 21:08:07,007 - lava_callback - ERROR - Error processing callback for node 69c05a32c3d73288b0d087e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42706 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:09:01,730 - lava_callback - INFO - Received callback for node 69c2bfbe6778148209fd35d0 INFO: 172.21.0.1:50288 - "POST /node/69c2bfbe6778148209fd35d0 HTTP/1.0" 202 Accepted 2026-03-24 21:09:01,736 - lava_callback - INFO - Starting async job submit for node 69c2bfbe6778148209fd35d0 2026-03-24 21:09:01,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:01,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfbe6778148209fd35d0 HTTP/1.1" 200 None 2026-03-24 21:09:01,778 - lava_callback - DEBUG - Processing node 69c2bfbe6778148209fd35d0: baseline-x86 2026-03-24 21:09:02,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:09:02,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:09:02,241 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfbe6778148209fd35d0/log.txt.gz 2026-03-24 21:09:02,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:09:02,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:09:02,593 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfbe6778148209fd35d0/lava-logs.yaml 2026-03-24 21:09:02,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:09:02,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:09:02,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c2bfbe6778148209fd35d0/lava_callback.json.gz 2026-03-24 21:09:02,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:02,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2bfbe6778148209fd35d0 HTTP/1.1" 200 None 2026-03-24 21:09:02,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:02,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b97f6778148209fd3394 HTTP/1.1" 200 None 2026-03-24 21:09:02,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:02,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2bfbe6778148209fd35d0 HTTP/1.1" 200 None 2026-03-24 21:09:02,854 - lava_callback - INFO - Completed processing callback for node 69c2bfbe6778148209fd35d0 2026-03-24 21:09:05,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:05,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57908 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:09:26,967 - lava_callback - INFO - Received callback for node 69c05a83c3d73288b0d087f4 INFO: 172.21.0.1:57910 - "POST /node/69c05a83c3d73288b0d087f4 HTTP/1.0" 202 Accepted 2026-03-24 21:09:26,973 - lava_callback - INFO - Starting async job submit for node 69c05a83c3d73288b0d087f4 2026-03-24 21:09:26,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:26,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05a83c3d73288b0d087f4 HTTP/1.1" 200 None 2026-03-24 21:09:26,988 - lava_callback - DEBUG - Processing node 69c05a83c3d73288b0d087f4: baseline-arm64-clabbe 2026-03-24 21:09:26,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05a83c3d73288b0d087f4 2026-03-24 21:09:26,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:27,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05a83c3d73288b0d087f4 HTTP/1.1" 200 None 2026-03-24 21:09:27,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:27,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 21:09:27,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:09:27,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05a83c3d73288b0d087f4 HTTP/1.1" 200 None 2026-03-24 21:09:27,045 - lava_callback - ERROR - Error processing callback for node 69c05a83c3d73288b0d087f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47980 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:10:30,097 - lava_callback - INFO - Received callback for node 69c2f6ec6778148209fd6708 INFO: 172.21.0.1:58512 - "POST /node/69c2f6ec6778148209fd6708 HTTP/1.0" 202 Accepted 2026-03-24 21:10:30,103 - lava_callback - INFO - Starting async job submit for node 69c2f6ec6778148209fd6708 2026-03-24 21:10:30,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:30,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f6ec6778148209fd6708 HTTP/1.1" 200 None 2026-03-24 21:10:30,135 - lava_callback - DEBUG - Processing node 69c2f6ec6778148209fd6708: baseline-x86 2026-03-24 21:10:30,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:10:30,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:10:30,608 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c2f6ec6778148209fd6708/log.txt.gz 2026-03-24 21:10:30,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:10:31,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:10:31,006 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c2f6ec6778148209fd6708/lava-logs.yaml 2026-03-24 21:10:31,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:10:31,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:10:31,173 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c2f6ec6778148209fd6708/lava_callback.json.gz 2026-03-24 21:10:31,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:31,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2f6ec6778148209fd6708 HTTP/1.1" 200 None 2026-03-24 21:10:31,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:31,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248fb4a2c5046eb87f8bb HTTP/1.1" 200 None 2026-03-24 21:10:31,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:31,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2f6ec6778148209fd6708 HTTP/1.1" 200 None 2026-03-24 21:10:31,258 - lava_callback - INFO - Completed processing callback for node 69c2f6ec6778148209fd6708 2026-03-24 21:10:35,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:35,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35232 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:10:47,280 - lava_callback - INFO - Received callback for node 69c05ad3c3d73288b0d08802 INFO: 172.21.0.1:35234 - "POST /node/69c05ad3c3d73288b0d08802 HTTP/1.0" 202 Accepted 2026-03-24 21:10:47,286 - lava_callback - INFO - Starting async job submit for node 69c05ad3c3d73288b0d08802 2026-03-24 21:10:47,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:47,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05ad3c3d73288b0d08802 HTTP/1.1" 200 None 2026-03-24 21:10:47,302 - lava_callback - DEBUG - Processing node 69c05ad3c3d73288b0d08802: baseline-arm64-clabbe 2026-03-24 21:10:47,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05ad3c3d73288b0d08802 2026-03-24 21:10:47,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:47,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05ad3c3d73288b0d08802 HTTP/1.1" 200 None 2026-03-24 21:10:47,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:47,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 21:10:47,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:10:47,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05ad3c3d73288b0d08802 HTTP/1.1" 200 None 2026-03-24 21:10:47,380 - lava_callback - ERROR - Error processing callback for node 69c05ad3c3d73288b0d08802: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39734 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:12:07,284 - lava_callback - INFO - Received callback for node 69c05b23c3d73288b0d0880d INFO: 172.21.0.1:39738 - "POST /node/69c05b23c3d73288b0d0880d HTTP/1.0" 202 Accepted 2026-03-24 21:12:07,289 - lava_callback - INFO - Starting async job submit for node 69c05b23c3d73288b0d0880d 2026-03-24 21:12:07,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:12:07,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05b23c3d73288b0d0880d HTTP/1.1" 200 None 2026-03-24 21:12:07,306 - lava_callback - DEBUG - Processing node 69c05b23c3d73288b0d0880d: baseline-arm64-clabbe 2026-03-24 21:12:07,307 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05b23c3d73288b0d0880d 2026-03-24 21:12:07,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:12:07,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05b23c3d73288b0d0880d HTTP/1.1" 200 None 2026-03-24 21:12:07,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:12:07,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 21:12:07,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:12:07,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05b23c3d73288b0d0880d HTTP/1.1" 200 None 2026-03-24 21:12:07,372 - lava_callback - ERROR - Error processing callback for node 69c05b23c3d73288b0d0880d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49916 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:13:19,732 - lava_callback - INFO - Received callback for node 69c05b6dc3d73288b0d0881d INFO: 172.21.0.1:60562 - "POST /node/69c05b6dc3d73288b0d0881d HTTP/1.0" 202 Accepted 2026-03-24 21:13:19,737 - lava_callback - INFO - Starting async job submit for node 69c05b6dc3d73288b0d0881d 2026-03-24 21:13:19,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:13:19,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05b6dc3d73288b0d0881d HTTP/1.1" 200 None 2026-03-24 21:13:19,753 - lava_callback - DEBUG - Processing node 69c05b6dc3d73288b0d0881d: baseline-arm64-baylibre 2026-03-24 21:13:19,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05b6dc3d73288b0d0881d 2026-03-24 21:13:19,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:13:19,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05b6dc3d73288b0d0881d HTTP/1.1" 200 None 2026-03-24 21:13:19,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:13:19,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:13:19,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:13:19,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05b6dc3d73288b0d0881d HTTP/1.1" 200 None 2026-03-24 21:13:19,813 - lava_callback - ERROR - Error processing callback for node 69c05b6dc3d73288b0d0881d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35216 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:14:47,246 - lava_callback - INFO - Received callback for node 69c05bb3c3d73288b0d08825 INFO: 172.21.0.1:35220 - "POST /node/69c05bb3c3d73288b0d08825 HTTP/1.0" 202 Accepted 2026-03-24 21:14:47,251 - lava_callback - INFO - Starting async job submit for node 69c05bb3c3d73288b0d08825 2026-03-24 21:14:47,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:14:47,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05bb3c3d73288b0d08825 HTTP/1.1" 200 None 2026-03-24 21:14:47,274 - lava_callback - DEBUG - Processing node 69c05bb3c3d73288b0d08825: baseline-arm-clabbe 2026-03-24 21:14:47,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05bb3c3d73288b0d08825 2026-03-24 21:14:47,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:14:47,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05bb3c3d73288b0d08825 HTTP/1.1" 200 None 2026-03-24 21:14:47,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:14:47,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 21:14:47,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:14:47,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05bb3c3d73288b0d08825 HTTP/1.1" 200 None 2026-03-24 21:14:47,348 - lava_callback - ERROR - Error processing callback for node 69c05bb3c3d73288b0d08825: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49058 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:15:07,393 - lava_callback - INFO - Received callback for node 69c05bd6c3d73288b0d0882f INFO: 172.21.0.1:49072 - "POST /node/69c05bd6c3d73288b0d0882f HTTP/1.0" 202 Accepted 2026-03-24 21:15:07,399 - lava_callback - INFO - Starting async job submit for node 69c05bd6c3d73288b0d0882f 2026-03-24 21:15:07,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:07,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05bd6c3d73288b0d0882f HTTP/1.1" 200 None 2026-03-24 21:15:07,415 - lava_callback - DEBUG - Processing node 69c05bd6c3d73288b0d0882f: baseline-arm-clabbe 2026-03-24 21:15:07,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05bd6c3d73288b0d0882f 2026-03-24 21:15:07,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:07,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05bd6c3d73288b0d0882f HTTP/1.1" 200 None 2026-03-24 21:15:07,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:07,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 21:15:07,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:07,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05bd6c3d73288b0d0882f HTTP/1.1" 200 None 2026-03-24 21:15:07,472 - lava_callback - ERROR - Error processing callback for node 69c05bd6c3d73288b0d0882f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49148 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:15:47,269 - lava_callback - INFO - Received callback for node 69c05bfac3d73288b0d08839 INFO: 172.21.0.1:49162 - "POST /node/69c05bfac3d73288b0d08839 HTTP/1.0" 202 Accepted 2026-03-24 21:15:47,274 - lava_callback - INFO - Starting async job submit for node 69c05bfac3d73288b0d08839 2026-03-24 21:15:47,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:47,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05bfac3d73288b0d08839 HTTP/1.1" 200 None 2026-03-24 21:15:47,290 - lava_callback - DEBUG - Processing node 69c05bfac3d73288b0d08839: baseline-arm-clabbe 2026-03-24 21:15:47,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05bfac3d73288b0d08839 2026-03-24 21:15:47,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:47,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05bfac3d73288b0d08839 HTTP/1.1" 200 None 2026-03-24 21:15:47,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:47,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 21:15:47,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:15:47,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05bfac3d73288b0d08839 HTTP/1.1" 200 None 2026-03-24 21:15:47,343 - lava_callback - ERROR - Error processing callback for node 69c05bfac3d73288b0d08839: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55342 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:16:47,380 - lava_callback - INFO - Received callback for node 69c05c2dc3d73288b0d08847 INFO: 172.21.0.1:55346 - "POST /node/69c05c2dc3d73288b0d08847 HTTP/1.0" 202 Accepted 2026-03-24 21:16:47,386 - lava_callback - INFO - Starting async job submit for node 69c05c2dc3d73288b0d08847 2026-03-24 21:16:47,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:16:47,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05c2dc3d73288b0d08847 HTTP/1.1" 200 None 2026-03-24 21:16:47,407 - lava_callback - DEBUG - Processing node 69c05c2dc3d73288b0d08847: baseline-arm64-clabbe 2026-03-24 21:16:47,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05c2dc3d73288b0d08847 2026-03-24 21:16:47,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:16:47,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05c2dc3d73288b0d08847 HTTP/1.1" 200 None 2026-03-24 21:16:47,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:16:47,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:16:47,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:16:47,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05c2dc3d73288b0d08847 HTTP/1.1" 200 None 2026-03-24 21:16:47,469 - lava_callback - ERROR - Error processing callback for node 69c05c2dc3d73288b0d08847: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56094 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:18:07,390 - lava_callback - INFO - Received callback for node 69c05c7dc3d73288b0d08858 INFO: 172.21.0.1:56104 - "POST /node/69c05c7dc3d73288b0d08858 HTTP/1.0" 202 Accepted 2026-03-24 21:18:07,395 - lava_callback - INFO - Starting async job submit for node 69c05c7dc3d73288b0d08858 2026-03-24 21:18:07,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:18:07,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05c7dc3d73288b0d08858 HTTP/1.1" 200 None 2026-03-24 21:18:07,415 - lava_callback - DEBUG - Processing node 69c05c7dc3d73288b0d08858: baseline-arm64-clabbe 2026-03-24 21:18:07,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05c7dc3d73288b0d08858 2026-03-24 21:18:07,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:18:07,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05c7dc3d73288b0d08858 HTTP/1.1" 200 None 2026-03-24 21:18:07,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:18:07,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:18:07,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:18:07,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05c7dc3d73288b0d08858 HTTP/1.1" 200 None 2026-03-24 21:18:07,473 - lava_callback - ERROR - Error processing callback for node 69c05c7dc3d73288b0d08858: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56676 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:19:27,445 - lava_callback - INFO - Received callback for node 69c05cccc3d73288b0d08865 INFO: 172.21.0.1:56686 - "POST /node/69c05cccc3d73288b0d08865 HTTP/1.0" 202 Accepted 2026-03-24 21:19:27,450 - lava_callback - INFO - Starting async job submit for node 69c05cccc3d73288b0d08865 2026-03-24 21:19:27,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:19:27,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05cccc3d73288b0d08865 HTTP/1.1" 200 None 2026-03-24 21:19:27,484 - lava_callback - DEBUG - Processing node 69c05cccc3d73288b0d08865: baseline-arm64-clabbe 2026-03-24 21:19:27,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05cccc3d73288b0d08865 2026-03-24 21:19:27,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:19:27,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05cccc3d73288b0d08865 HTTP/1.1" 200 None 2026-03-24 21:19:27,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:19:27,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:19:27,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:19:27,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05cccc3d73288b0d08865 HTTP/1.1" 200 None 2026-03-24 21:19:27,549 - lava_callback - ERROR - Error processing callback for node 69c05cccc3d73288b0d08865: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:21:40,220 - lava_callback - INFO - Received callback for node 69c05d5bc3d73288b0d08887 INFO: 172.21.0.1:40602 - "POST /node/69c05d5bc3d73288b0d08887 HTTP/1.0" 202 Accepted 2026-03-24 21:21:40,228 - lava_callback - INFO - Starting async job submit for node 69c05d5bc3d73288b0d08887 2026-03-24 21:21:40,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:21:40,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05d5bc3d73288b0d08887 HTTP/1.1" 200 None 2026-03-24 21:21:40,244 - lava_callback - DEBUG - Processing node 69c05d5bc3d73288b0d08887: baseline-arm64-baylibre 2026-03-24 21:21:40,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05d5bc3d73288b0d08887 2026-03-24 21:21:40,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:21:40,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05d5bc3d73288b0d08887 HTTP/1.1" 200 None 2026-03-24 21:21:40,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:21:40,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:21:40,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:21:40,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05d5bc3d73288b0d08887 HTTP/1.1" 200 None 2026-03-24 21:21:40,304 - lava_callback - ERROR - Error processing callback for node 69c05d5bc3d73288b0d08887: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:22:40,140 - lava_callback - INFO - Received callback for node 69c05d9cc3d73288b0d08894 INFO: 172.21.0.1:49422 - "POST /node/69c05d9cc3d73288b0d08894 HTTP/1.0" 202 Accepted 2026-03-24 21:22:40,146 - lava_callback - INFO - Starting async job submit for node 69c05d9cc3d73288b0d08894 2026-03-24 21:22:40,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:22:40,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05d9cc3d73288b0d08894 HTTP/1.1" 200 None 2026-03-24 21:22:40,187 - lava_callback - DEBUG - Processing node 69c05d9cc3d73288b0d08894: baseline-arm-baylibre 2026-03-24 21:22:40,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05d9cc3d73288b0d08894 2026-03-24 21:22:40,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:22:40,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05d9cc3d73288b0d08894 HTTP/1.1" 200 None 2026-03-24 21:22:40,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:22:40,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:22:40,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:22:40,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05d9cc3d73288b0d08894 HTTP/1.1" 200 None 2026-03-24 21:22:40,242 - lava_callback - ERROR - Error processing callback for node 69c05d9cc3d73288b0d08894: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37224 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:23:20,359 - lava_callback - INFO - Received callback for node 69c05dc0c3d73288b0d088a3 INFO: 172.21.0.1:57800 - "POST /node/69c05dc0c3d73288b0d088a3 HTTP/1.0" 202 Accepted 2026-03-24 21:23:20,365 - lava_callback - INFO - Starting async job submit for node 69c05dc0c3d73288b0d088a3 2026-03-24 21:23:20,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:23:20,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05dc0c3d73288b0d088a3 HTTP/1.1" 200 None 2026-03-24 21:23:20,386 - lava_callback - DEBUG - Processing node 69c05dc0c3d73288b0d088a3: baseline-arm-baylibre 2026-03-24 21:23:20,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05dc0c3d73288b0d088a3 2026-03-24 21:23:20,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:23:20,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05dc0c3d73288b0d088a3 HTTP/1.1" 200 None 2026-03-24 21:23:20,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:23:20,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:23:20,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:23:20,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05dc0c3d73288b0d088a3 HTTP/1.1" 200 None 2026-03-24 21:23:20,446 - lava_callback - ERROR - Error processing callback for node 69c05dc0c3d73288b0d088a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51800 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:24:00,212 - lava_callback - INFO - Received callback for node 69c05de4c3d73288b0d088ae INFO: 172.21.0.1:34380 - "POST /node/69c05de4c3d73288b0d088ae HTTP/1.0" 202 Accepted 2026-03-24 21:24:00,217 - lava_callback - INFO - Starting async job submit for node 69c05de4c3d73288b0d088ae 2026-03-24 21:24:00,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:00,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05de4c3d73288b0d088ae HTTP/1.1" 200 None 2026-03-24 21:24:00,233 - lava_callback - DEBUG - Processing node 69c05de4c3d73288b0d088ae: baseline-arm-baylibre 2026-03-24 21:24:00,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05de4c3d73288b0d088ae 2026-03-24 21:24:00,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:00,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05de4c3d73288b0d088ae HTTP/1.1" 200 None 2026-03-24 21:24:00,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:00,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:24:00,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:00,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05de4c3d73288b0d088ae HTTP/1.1" 200 None 2026-03-24 21:24:00,324 - lava_callback - ERROR - Error processing callback for node 69c05de4c3d73288b0d088ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48398 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:24:47,702 - lava_callback - INFO - Received callback for node 69c05e18c3d73288b0d088b8 INFO: 172.21.0.1:48410 - "POST /node/69c05e18c3d73288b0d088b8 HTTP/1.0" 202 Accepted 2026-03-24 21:24:47,707 - lava_callback - INFO - Starting async job submit for node 69c05e18c3d73288b0d088b8 2026-03-24 21:24:47,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:47,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05e18c3d73288b0d088b8 HTTP/1.1" 200 None 2026-03-24 21:24:47,731 - lava_callback - DEBUG - Processing node 69c05e18c3d73288b0d088b8: baseline-arm64-clabbe 2026-03-24 21:24:47,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05e18c3d73288b0d088b8 2026-03-24 21:24:47,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:47,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05e18c3d73288b0d088b8 HTTP/1.1" 200 None 2026-03-24 21:24:47,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:47,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:24:47,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:24:47,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05e18c3d73288b0d088b8 HTTP/1.1" 200 None 2026-03-24 21:24:47,782 - lava_callback - ERROR - Error processing callback for node 69c05e18c3d73288b0d088b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51192 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:26:07,759 - lava_callback - INFO - Received callback for node 69c05e66c3d73288b0d088cb INFO: 172.21.0.1:51196 - "POST /node/69c05e66c3d73288b0d088cb HTTP/1.0" 202 Accepted 2026-03-24 21:26:07,765 - lava_callback - INFO - Starting async job submit for node 69c05e66c3d73288b0d088cb 2026-03-24 21:26:07,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:26:07,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05e66c3d73288b0d088cb HTTP/1.1" 200 None 2026-03-24 21:26:07,781 - lava_callback - DEBUG - Processing node 69c05e66c3d73288b0d088cb: baseline-arm64-clabbe 2026-03-24 21:26:07,781 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05e66c3d73288b0d088cb 2026-03-24 21:26:07,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:26:07,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05e66c3d73288b0d088cb HTTP/1.1" 200 None 2026-03-24 21:26:07,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:26:07,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:26:07,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:26:07,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05e66c3d73288b0d088cb HTTP/1.1" 200 None 2026-03-24 21:26:07,863 - lava_callback - ERROR - Error processing callback for node 69c05e66c3d73288b0d088cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:28:27,848 - lava_callback - INFO - Received callback for node 69c05eedc3d73288b0d088f2 INFO: 172.21.0.1:53548 - "POST /node/69c05eedc3d73288b0d088f2 HTTP/1.0" 202 Accepted 2026-03-24 21:28:27,854 - lava_callback - INFO - Starting async job submit for node 69c05eedc3d73288b0d088f2 2026-03-24 21:28:27,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:28:27,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05eedc3d73288b0d088f2 HTTP/1.1" 200 None 2026-03-24 21:28:27,870 - lava_callback - DEBUG - Processing node 69c05eedc3d73288b0d088f2: baseline-arm-clabbe 2026-03-24 21:28:27,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05eedc3d73288b0d088f2 2026-03-24 21:28:27,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:28:27,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05eedc3d73288b0d088f2 HTTP/1.1" 200 None 2026-03-24 21:28:27,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:28:27,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:28:27,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:28:27,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05eedc3d73288b0d088f2 HTTP/1.1" 200 None 2026-03-24 21:28:27,933 - lava_callback - ERROR - Error processing callback for node 69c05eedc3d73288b0d088f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57768 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:29:07,820 - lava_callback - INFO - Received callback for node 69c05f11c3d73288b0d08902 INFO: 172.21.0.1:57778 - "POST /node/69c05f11c3d73288b0d08902 HTTP/1.0" 202 Accepted 2026-03-24 21:29:07,825 - lava_callback - INFO - Starting async job submit for node 69c05f11c3d73288b0d08902 2026-03-24 21:29:07,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:07,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05f11c3d73288b0d08902 HTTP/1.1" 200 None 2026-03-24 21:29:07,842 - lava_callback - DEBUG - Processing node 69c05f11c3d73288b0d08902: baseline-arm-clabbe 2026-03-24 21:29:07,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05f11c3d73288b0d08902 2026-03-24 21:29:07,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:07,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05f11c3d73288b0d08902 HTTP/1.1" 200 None 2026-03-24 21:29:07,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:07,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:29:07,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:07,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05f11c3d73288b0d08902 HTTP/1.1" 200 None 2026-03-24 21:29:07,913 - lava_callback - ERROR - Error processing callback for node 69c05f11c3d73288b0d08902: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:29:48,057 - lava_callback - INFO - Received callback for node 69c05f36c3d73288b0d0890f INFO: 172.21.0.1:53970 - "POST /node/69c05f36c3d73288b0d0890f HTTP/1.0" 202 Accepted 2026-03-24 21:29:48,062 - lava_callback - INFO - Starting async job submit for node 69c05f36c3d73288b0d0890f 2026-03-24 21:29:48,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:48,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05f36c3d73288b0d0890f HTTP/1.1" 200 None 2026-03-24 21:29:48,078 - lava_callback - DEBUG - Processing node 69c05f36c3d73288b0d0890f: baseline-arm-clabbe 2026-03-24 21:29:48,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05f36c3d73288b0d0890f 2026-03-24 21:29:48,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:48,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05f36c3d73288b0d0890f HTTP/1.1" 200 None 2026-03-24 21:29:48,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:48,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:29:48,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:29:48,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05f36c3d73288b0d0890f HTTP/1.1" 200 None 2026-03-24 21:29:48,160 - lava_callback - ERROR - Error processing callback for node 69c05f36c3d73288b0d0890f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:30:27,953 - lava_callback - INFO - Received callback for node 69c05f69c3d73288b0d0891d INFO: 172.21.0.1:51750 - "POST /node/69c05f69c3d73288b0d0891d HTTP/1.0" 202 Accepted 2026-03-24 21:30:27,958 - lava_callback - INFO - Starting async job submit for node 69c05f69c3d73288b0d0891d 2026-03-24 21:30:27,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:30:27,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05f69c3d73288b0d0891d HTTP/1.1" 200 None 2026-03-24 21:30:27,974 - lava_callback - DEBUG - Processing node 69c05f69c3d73288b0d0891d: baseline-arm64-clabbe 2026-03-24 21:30:27,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05f69c3d73288b0d0891d 2026-03-24 21:30:27,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:30:27,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05f69c3d73288b0d0891d HTTP/1.1" 200 None 2026-03-24 21:30:27,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:30:28,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:30:28,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:30:28,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05f69c3d73288b0d0891d HTTP/1.1" 200 None 2026-03-24 21:30:28,033 - lava_callback - ERROR - Error processing callback for node 69c05f69c3d73288b0d0891d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:31:27,950 - lava_callback - INFO - Received callback for node 69c05fa8c3d73288b0d08925 INFO: 172.21.0.1:33000 - "POST /node/69c05fa8c3d73288b0d08925 HTTP/1.0" 202 Accepted 2026-03-24 21:31:27,955 - lava_callback - INFO - Starting async job submit for node 69c05fa8c3d73288b0d08925 2026-03-24 21:31:27,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:31:27,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05fa8c3d73288b0d08925 HTTP/1.1" 200 None 2026-03-24 21:31:27,971 - lava_callback - DEBUG - Processing node 69c05fa8c3d73288b0d08925: baseline-arm-clabbe 2026-03-24 21:31:27,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05fa8c3d73288b0d08925 2026-03-24 21:31:27,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:31:27,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05fa8c3d73288b0d08925 HTTP/1.1" 200 None 2026-03-24 21:31:27,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:31:28,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:31:28,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:31:28,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05fa8c3d73288b0d08925 HTTP/1.1" 200 None 2026-03-24 21:31:28,032 - lava_callback - ERROR - Error processing callback for node 69c05fa8c3d73288b0d08925: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59034 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:32:07,932 - lava_callback - INFO - Received callback for node 69c05fccc3d73288b0d08933 INFO: 172.21.0.1:59044 - "POST /node/69c05fccc3d73288b0d08933 HTTP/1.0" 202 Accepted 2026-03-24 21:32:07,938 - lava_callback - INFO - Starting async job submit for node 69c05fccc3d73288b0d08933 2026-03-24 21:32:07,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:32:07,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05fccc3d73288b0d08933 HTTP/1.1" 200 None 2026-03-24 21:32:07,953 - lava_callback - DEBUG - Processing node 69c05fccc3d73288b0d08933: baseline-arm-clabbe 2026-03-24 21:32:07,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c05fccc3d73288b0d08933 2026-03-24 21:32:07,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:32:07,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c05fccc3d73288b0d08933 HTTP/1.1" 200 None 2026-03-24 21:32:07,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:32:07,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 21:32:07,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:32:08,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c05fccc3d73288b0d08933 HTTP/1.1" 200 None 2026-03-24 21:32:08,015 - lava_callback - ERROR - Error processing callback for node 69c05fccc3d73288b0d08933: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51160 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:33:08,048 - lava_callback - INFO - Received callback for node 69c06000c3d73288b0d0894a INFO: 172.21.0.1:51174 - "POST /node/69c06000c3d73288b0d0894a HTTP/1.0" 202 Accepted 2026-03-24 21:33:08,054 - lava_callback - INFO - Starting async job submit for node 69c06000c3d73288b0d0894a 2026-03-24 21:33:08,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:33:08,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06000c3d73288b0d0894a HTTP/1.1" 200 None 2026-03-24 21:33:08,071 - lava_callback - DEBUG - Processing node 69c06000c3d73288b0d0894a: baseline-arm64-clabbe 2026-03-24 21:33:08,071 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06000c3d73288b0d0894a 2026-03-24 21:33:08,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:33:08,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06000c3d73288b0d0894a HTTP/1.1" 200 None 2026-03-24 21:33:08,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:33:08,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:33:08,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:33:08,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06000c3d73288b0d0894a HTTP/1.1" 200 None 2026-03-24 21:33:08,131 - lava_callback - ERROR - Error processing callback for node 69c06000c3d73288b0d0894a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45112 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:34:28,095 - lava_callback - INFO - Received callback for node 69c0604ec3d73288b0d08953 INFO: 172.21.0.1:45116 - "POST /node/69c0604ec3d73288b0d08953 HTTP/1.0" 202 Accepted 2026-03-24 21:34:28,100 - lava_callback - INFO - Starting async job submit for node 69c0604ec3d73288b0d08953 2026-03-24 21:34:28,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:28,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0604ec3d73288b0d08953 HTTP/1.1" 200 None 2026-03-24 21:34:28,116 - lava_callback - DEBUG - Processing node 69c0604ec3d73288b0d08953: baseline-arm64-clabbe 2026-03-24 21:34:28,116 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0604ec3d73288b0d08953 2026-03-24 21:34:28,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:28,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0604ec3d73288b0d08953 HTTP/1.1" 200 None 2026-03-24 21:34:28,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:28,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:34:28,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:28,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0604ec3d73288b0d08953 HTTP/1.1" 200 None 2026-03-24 21:34:28,173 - lava_callback - ERROR - Error processing callback for node 69c0604ec3d73288b0d08953: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 21:34:30,595 - lava_callback - INFO - Received callback for node 69c2d8336778148209fd6459 INFO: 172.21.0.1:33350 - "POST /node/69c2d8336778148209fd6459 HTTP/1.0" 202 Accepted 2026-03-24 21:34:30,641 - lava_callback - INFO - Starting async job submit for node 69c2d8336778148209fd6459 2026-03-24 21:34:30,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:30,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8336778148209fd6459 HTTP/1.1" 200 None 2026-03-24 21:34:30,680 - lava_callback - DEBUG - Processing node 69c2d8336778148209fd6459: baseline-arm-pengutronix 2026-03-24 21:34:30,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:34:30,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:34:30,964 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2d8336778148209fd6459/log.txt.gz 2026-03-24 21:34:31,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:34:31,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:34:31,246 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2d8336778148209fd6459/lava-logs.yaml 2026-03-24 21:34:31,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 21:34:31,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 21:34:31,398 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c2d8336778148209fd6459/lava_callback.json.gz DEBUG: Setup failed for 69c2d8336778148209fd6459. Transitting job node result: incomplete -> fail 2026-03-24 21:34:31,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:31,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2d8336778148209fd6459 HTTP/1.1" 200 None 2026-03-24 21:34:31,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:31,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 21:34:31,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:31,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2d8336778148209fd6459 HTTP/1.1" 200 None 2026-03-24 21:34:31,589 - lava_callback - INFO - Completed processing callback for node 69c2d8336778148209fd6459 2026-03-24 21:34:35,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:34:35,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55964 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:35:48,158 - lava_callback - INFO - Received callback for node 69c0609ec3d73288b0d0895f INFO: 172.21.0.1:55978 - "POST /node/69c0609ec3d73288b0d0895f HTTP/1.0" 202 Accepted 2026-03-24 21:35:48,164 - lava_callback - INFO - Starting async job submit for node 69c0609ec3d73288b0d0895f 2026-03-24 21:35:48,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:35:48,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0609ec3d73288b0d0895f HTTP/1.1" 200 None 2026-03-24 21:35:48,180 - lava_callback - DEBUG - Processing node 69c0609ec3d73288b0d0895f: baseline-arm64-clabbe 2026-03-24 21:35:48,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0609ec3d73288b0d0895f 2026-03-24 21:35:48,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:35:48,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0609ec3d73288b0d0895f HTTP/1.1" 200 None 2026-03-24 21:35:48,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:35:48,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:35:48,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:35:48,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0609ec3d73288b0d0895f HTTP/1.1" 200 None 2026-03-24 21:35:48,236 - lava_callback - ERROR - Error processing callback for node 69c0609ec3d73288b0d0895f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43076 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:37:08,219 - lava_callback - INFO - Received callback for node 69c060efc3d73288b0d0896f INFO: 172.21.0.1:43092 - "POST /node/69c060efc3d73288b0d0896f HTTP/1.0" 202 Accepted 2026-03-24 21:37:08,225 - lava_callback - INFO - Starting async job submit for node 69c060efc3d73288b0d0896f 2026-03-24 21:37:08,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:37:08,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c060efc3d73288b0d0896f HTTP/1.1" 200 None 2026-03-24 21:37:08,241 - lava_callback - DEBUG - Processing node 69c060efc3d73288b0d0896f: baseline-arm64-clabbe 2026-03-24 21:37:08,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c060efc3d73288b0d0896f 2026-03-24 21:37:08,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:37:08,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c060efc3d73288b0d0896f HTTP/1.1" 200 None 2026-03-24 21:37:08,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:37:08,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:37:08,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:37:08,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c060efc3d73288b0d0896f HTTP/1.1" 200 None 2026-03-24 21:37:08,301 - lava_callback - ERROR - Error processing callback for node 69c060efc3d73288b0d0896f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35114 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:38:01,155 - lava_callback - INFO - Received callback for node 69c06137c3d73288b0d0897a INFO: 172.21.0.1:55202 - "POST /node/69c06137c3d73288b0d0897a HTTP/1.0" 202 Accepted 2026-03-24 21:38:01,160 - lava_callback - INFO - Starting async job submit for node 69c06137c3d73288b0d0897a 2026-03-24 21:38:01,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:38:01,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06137c3d73288b0d0897a HTTP/1.1" 200 None 2026-03-24 21:38:01,176 - lava_callback - DEBUG - Processing node 69c06137c3d73288b0d0897a: baseline-arm64-baylibre 2026-03-24 21:38:01,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06137c3d73288b0d0897a 2026-03-24 21:38:01,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:38:01,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06137c3d73288b0d0897a HTTP/1.1" 200 None 2026-03-24 21:38:01,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:38:01,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:38:01,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:38:01,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06137c3d73288b0d0897a HTTP/1.1" 200 None 2026-03-24 21:38:01,235 - lava_callback - ERROR - Error processing callback for node 69c06137c3d73288b0d0897a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37208 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:39:28,310 - lava_callback - INFO - Received callback for node 69c0618bc3d73288b0d0898a INFO: 172.21.0.1:37210 - "POST /node/69c0618bc3d73288b0d0898a HTTP/1.0" 202 Accepted 2026-03-24 21:39:28,315 - lava_callback - INFO - Starting async job submit for node 69c0618bc3d73288b0d0898a 2026-03-24 21:39:28,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:39:28,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0618bc3d73288b0d0898a HTTP/1.1" 200 None 2026-03-24 21:39:28,332 - lava_callback - DEBUG - Processing node 69c0618bc3d73288b0d0898a: baseline-arm64-clabbe 2026-03-24 21:39:28,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0618bc3d73288b0d0898a 2026-03-24 21:39:28,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:39:28,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0618bc3d73288b0d0898a HTTP/1.1" 200 None 2026-03-24 21:39:28,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:39:28,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:39:28,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:39:28,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0618bc3d73288b0d0898a HTTP/1.1" 200 None 2026-03-24 21:39:28,398 - lava_callback - ERROR - Error processing callback for node 69c0618bc3d73288b0d0898a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44424 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:40:48,417 - lava_callback - INFO - Received callback for node 69c061dac3d73288b0d08994 INFO: 172.21.0.1:44434 - "POST /node/69c061dac3d73288b0d08994 HTTP/1.0" 202 Accepted 2026-03-24 21:40:48,423 - lava_callback - INFO - Starting async job submit for node 69c061dac3d73288b0d08994 2026-03-24 21:40:48,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:40:48,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c061dac3d73288b0d08994 HTTP/1.1" 200 None 2026-03-24 21:40:48,439 - lava_callback - DEBUG - Processing node 69c061dac3d73288b0d08994: baseline-arm64-clabbe 2026-03-24 21:40:48,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c061dac3d73288b0d08994 2026-03-24 21:40:48,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:40:48,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c061dac3d73288b0d08994 HTTP/1.1" 200 None 2026-03-24 21:40:48,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:40:48,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:40:48,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:40:48,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c061dac3d73288b0d08994 HTTP/1.1" 200 None 2026-03-24 21:40:48,538 - lava_callback - ERROR - Error processing callback for node 69c061dac3d73288b0d08994: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50876 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:42:08,430 - lava_callback - INFO - Received callback for node 69c06229c3d73288b0d089a0 INFO: 172.21.0.1:50886 - "POST /node/69c06229c3d73288b0d089a0 HTTP/1.0" 202 Accepted 2026-03-24 21:42:08,438 - lava_callback - INFO - Starting async job submit for node 69c06229c3d73288b0d089a0 2026-03-24 21:42:08,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:42:08,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06229c3d73288b0d089a0 HTTP/1.1" 200 None 2026-03-24 21:42:08,452 - lava_callback - DEBUG - Processing node 69c06229c3d73288b0d089a0: baseline-arm64-clabbe 2026-03-24 21:42:08,452 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06229c3d73288b0d089a0 2026-03-24 21:42:08,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:42:08,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06229c3d73288b0d089a0 HTTP/1.1" 200 None 2026-03-24 21:42:08,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:42:08,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:42:08,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:42:08,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06229c3d73288b0d089a0 HTTP/1.1" 200 None 2026-03-24 21:42:08,509 - lava_callback - ERROR - Error processing callback for node 69c06229c3d73288b0d089a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46912 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:43:28,489 - lava_callback - INFO - Received callback for node 69c06278c3d73288b0d089af INFO: 172.21.0.1:46916 - "POST /node/69c06278c3d73288b0d089af HTTP/1.0" 202 Accepted 2026-03-24 21:43:28,496 - lava_callback - INFO - Starting async job submit for node 69c06278c3d73288b0d089af 2026-03-24 21:43:28,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:43:28,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06278c3d73288b0d089af HTTP/1.1" 200 None 2026-03-24 21:43:28,518 - lava_callback - DEBUG - Processing node 69c06278c3d73288b0d089af: baseline-arm64-clabbe 2026-03-24 21:43:28,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06278c3d73288b0d089af 2026-03-24 21:43:28,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:43:28,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06278c3d73288b0d089af HTTP/1.1" 200 None 2026-03-24 21:43:28,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:43:28,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:43:28,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:43:28,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06278c3d73288b0d089af HTTP/1.1" 200 None 2026-03-24 21:43:28,591 - lava_callback - ERROR - Error processing callback for node 69c06278c3d73288b0d089af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47464 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:44:48,516 - lava_callback - INFO - Received callback for node 69c062c6c3d73288b0d089bb INFO: 172.21.0.1:47470 - "POST /node/69c062c6c3d73288b0d089bb HTTP/1.0" 202 Accepted 2026-03-24 21:44:48,521 - lava_callback - INFO - Starting async job submit for node 69c062c6c3d73288b0d089bb 2026-03-24 21:44:48,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:44:48,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c062c6c3d73288b0d089bb HTTP/1.1" 200 None 2026-03-24 21:44:48,538 - lava_callback - DEBUG - Processing node 69c062c6c3d73288b0d089bb: baseline-arm64-clabbe 2026-03-24 21:44:48,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c062c6c3d73288b0d089bb 2026-03-24 21:44:48,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:44:48,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c062c6c3d73288b0d089bb HTTP/1.1" 200 None 2026-03-24 21:44:48,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:44:48,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:44:48,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:44:48,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c062c6c3d73288b0d089bb HTTP/1.1" 200 None 2026-03-24 21:44:48,590 - lava_callback - ERROR - Error processing callback for node 69c062c6c3d73288b0d089bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:46:08,547 - lava_callback - INFO - Received callback for node 69c06312c3d73288b0d089c7 INFO: 172.21.0.1:42268 - "POST /node/69c06312c3d73288b0d089c7 HTTP/1.0" 202 Accepted 2026-03-24 21:46:08,552 - lava_callback - INFO - Starting async job submit for node 69c06312c3d73288b0d089c7 2026-03-24 21:46:08,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:46:08,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06312c3d73288b0d089c7 HTTP/1.1" 200 None 2026-03-24 21:46:08,584 - lava_callback - DEBUG - Processing node 69c06312c3d73288b0d089c7: baseline-arm64-clabbe 2026-03-24 21:46:08,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06312c3d73288b0d089c7 2026-03-24 21:46:08,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:46:08,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06312c3d73288b0d089c7 HTTP/1.1" 200 None 2026-03-24 21:46:08,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:46:08,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:46:08,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:46:08,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06312c3d73288b0d089c7 HTTP/1.1" 200 None 2026-03-24 21:46:08,638 - lava_callback - ERROR - Error processing callback for node 69c06312c3d73288b0d089c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:47:28,702 - lava_callback - INFO - Received callback for node 69c06360c3d73288b0d089d3 INFO: 172.21.0.1:47982 - "POST /node/69c06360c3d73288b0d089d3 HTTP/1.0" 202 Accepted 2026-03-24 21:47:28,708 - lava_callback - INFO - Starting async job submit for node 69c06360c3d73288b0d089d3 2026-03-24 21:47:28,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:47:28,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06360c3d73288b0d089d3 HTTP/1.1" 200 None 2026-03-24 21:47:28,724 - lava_callback - DEBUG - Processing node 69c06360c3d73288b0d089d3: baseline-arm64-clabbe 2026-03-24 21:47:28,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06360c3d73288b0d089d3 2026-03-24 21:47:28,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:47:28,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06360c3d73288b0d089d3 HTTP/1.1" 200 None 2026-03-24 21:47:28,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:47:28,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:47:28,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:47:28,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06360c3d73288b0d089d3 HTTP/1.1" 200 None 2026-03-24 21:47:28,776 - lava_callback - ERROR - Error processing callback for node 69c06360c3d73288b0d089d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51666 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60432 - "GET / HTTP/1.0" 200 OK 2026-03-24 21:48:48,652 - lava_callback - INFO - Received callback for node 69c063afc3d73288b0d089dc INFO: 172.21.0.1:60448 - "POST /node/69c063afc3d73288b0d089dc HTTP/1.0" 202 Accepted 2026-03-24 21:48:48,657 - lava_callback - INFO - Starting async job submit for node 69c063afc3d73288b0d089dc 2026-03-24 21:48:48,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:48:48,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c063afc3d73288b0d089dc HTTP/1.1" 200 None 2026-03-24 21:48:48,674 - lava_callback - DEBUG - Processing node 69c063afc3d73288b0d089dc: baseline-arm64-clabbe 2026-03-24 21:48:48,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c063afc3d73288b0d089dc 2026-03-24 21:48:48,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:48:48,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c063afc3d73288b0d089dc HTTP/1.1" 200 None 2026-03-24 21:48:48,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:48:48,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 21:48:48,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 21:48:48,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c063afc3d73288b0d089dc HTTP/1.1" 200 None 2026-03-24 21:48:48,736 - lava_callback - ERROR - Error processing callback for node 69c063afc3d73288b0d089dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41734 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:39:04,532 - lava_callback - INFO - Received callback for node 69c06f7ec3d73288b0d08a34 INFO: 172.21.0.1:57996 - "POST /node/69c06f7ec3d73288b0d08a34 HTTP/1.0" 202 Accepted 2026-03-24 22:39:04,538 - lava_callback - INFO - Starting async job submit for node 69c06f7ec3d73288b0d08a34 2026-03-24 22:39:04,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:39:04,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06f7ec3d73288b0d08a34 HTTP/1.1" 200 None 2026-03-24 22:39:04,563 - lava_callback - DEBUG - Processing node 69c06f7ec3d73288b0d08a34: baseline-arm-baylibre 2026-03-24 22:39:04,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06f7ec3d73288b0d08a34 2026-03-24 22:39:04,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:39:04,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06f7ec3d73288b0d08a34 HTTP/1.1" 200 None 2026-03-24 22:39:04,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:39:04,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:39:04,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:39:04,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06f7ec3d73288b0d08a34 HTTP/1.1" 200 None 2026-03-24 22:39:04,622 - lava_callback - ERROR - Error processing callback for node 69c06f7ec3d73288b0d08a34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35000 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:40:04,458 - lava_callback - INFO - Received callback for node 69c06fbac3d73288b0d08a43 INFO: 172.21.0.1:49768 - "POST /node/69c06fbac3d73288b0d08a43 HTTP/1.0" 202 Accepted 2026-03-24 22:40:04,463 - lava_callback - INFO - Starting async job submit for node 69c06fbac3d73288b0d08a43 2026-03-24 22:40:04,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:04,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06fbac3d73288b0d08a43 HTTP/1.1" 200 None 2026-03-24 22:40:04,519 - lava_callback - DEBUG - Processing node 69c06fbac3d73288b0d08a43: baseline-arm-baylibre 2026-03-24 22:40:04,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06fbac3d73288b0d08a43 2026-03-24 22:40:04,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:04,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06fbac3d73288b0d08a43 HTTP/1.1" 200 None 2026-03-24 22:40:04,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:04,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:40:04,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:04,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06fbac3d73288b0d08a43 HTTP/1.1" 200 None 2026-03-24 22:40:04,615 - lava_callback - ERROR - Error processing callback for node 69c06fbac3d73288b0d08a43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49412 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:40:44,976 - lava_callback - INFO - Received callback for node 69c06fdfc3d73288b0d08a4e INFO: 172.21.0.1:38614 - "POST /node/69c06fdfc3d73288b0d08a4e HTTP/1.0" 202 Accepted 2026-03-24 22:40:44,982 - lava_callback - INFO - Starting async job submit for node 69c06fdfc3d73288b0d08a4e 2026-03-24 22:40:44,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:45,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06fdfc3d73288b0d08a4e HTTP/1.1" 200 None 2026-03-24 22:40:45,011 - lava_callback - DEBUG - Processing node 69c06fdfc3d73288b0d08a4e: baseline-arm-baylibre 2026-03-24 22:40:45,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c06fdfc3d73288b0d08a4e 2026-03-24 22:40:45,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:45,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c06fdfc3d73288b0d08a4e HTTP/1.1" 200 None 2026-03-24 22:40:45,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:45,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:40:45,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:40:45,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c06fdfc3d73288b0d08a4e HTTP/1.1" 200 None 2026-03-24 22:40:45,080 - lava_callback - ERROR - Error processing callback for node 69c06fdfc3d73288b0d08a4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55028 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:41:24,532 - lava_callback - INFO - Received callback for node 69c07002c3d73288b0d08a5a INFO: 172.21.0.1:54660 - "POST /node/69c07002c3d73288b0d08a5a HTTP/1.0" 202 Accepted 2026-03-24 22:41:24,542 - lava_callback - INFO - Starting async job submit for node 69c07002c3d73288b0d08a5a 2026-03-24 22:41:24,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:41:24,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07002c3d73288b0d08a5a HTTP/1.1" 200 None 2026-03-24 22:41:24,593 - lava_callback - DEBUG - Processing node 69c07002c3d73288b0d08a5a: baseline-arm-baylibre 2026-03-24 22:41:24,593 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07002c3d73288b0d08a5a 2026-03-24 22:41:24,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:41:24,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07002c3d73288b0d08a5a HTTP/1.1" 200 None 2026-03-24 22:41:24,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:41:24,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:41:24,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:41:24,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07002c3d73288b0d08a5a HTTP/1.1" 200 None 2026-03-24 22:41:24,712 - lava_callback - ERROR - Error processing callback for node 69c07002c3d73288b0d08a5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59044 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:42:04,638 - lava_callback - INFO - Received callback for node 69c07035c3d73288b0d08a67 INFO: 172.21.0.1:53502 - "POST /node/69c07035c3d73288b0d08a67 HTTP/1.0" 202 Accepted 2026-03-24 22:42:04,644 - lava_callback - INFO - Starting async job submit for node 69c07035c3d73288b0d08a67 2026-03-24 22:42:04,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:42:04,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07035c3d73288b0d08a67 HTTP/1.1" 200 None 2026-03-24 22:42:04,662 - lava_callback - DEBUG - Processing node 69c07035c3d73288b0d08a67: baseline-arm-baylibre 2026-03-24 22:42:04,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07035c3d73288b0d08a67 2026-03-24 22:42:04,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:42:04,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07035c3d73288b0d08a67 HTTP/1.1" 200 None 2026-03-24 22:42:04,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:42:04,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:42:04,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:42:04,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07035c3d73288b0d08a67 HTTP/1.1" 200 None 2026-03-24 22:42:04,729 - lava_callback - ERROR - Error processing callback for node 69c07035c3d73288b0d08a67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60404 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:44:04,853 - lava_callback - INFO - Received callback for node 69c070adc3d73288b0d08a7f INFO: 172.21.0.1:38082 - "POST /node/69c070adc3d73288b0d08a7f HTTP/1.0" 202 Accepted 2026-03-24 22:44:04,859 - lava_callback - INFO - Starting async job submit for node 69c070adc3d73288b0d08a7f 2026-03-24 22:44:04,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:04,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c070adc3d73288b0d08a7f HTTP/1.1" 200 None 2026-03-24 22:44:04,877 - lava_callback - DEBUG - Processing node 69c070adc3d73288b0d08a7f: baseline-arm-baylibre 2026-03-24 22:44:04,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c070adc3d73288b0d08a7f 2026-03-24 22:44:04,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:04,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c070adc3d73288b0d08a7f HTTP/1.1" 200 None 2026-03-24 22:44:04,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:04,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:44:04,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:04,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c070adc3d73288b0d08a7f HTTP/1.1" 200 None 2026-03-24 22:44:04,935 - lava_callback - ERROR - Error processing callback for node 69c070adc3d73288b0d08a7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:44:44,655 - lava_callback - INFO - Received callback for node 69c070cec3d73288b0d08a8b INFO: 172.21.0.1:43474 - "POST /node/69c070cec3d73288b0d08a8b HTTP/1.0" 202 Accepted 2026-03-24 22:44:44,661 - lava_callback - INFO - Starting async job submit for node 69c070cec3d73288b0d08a8b 2026-03-24 22:44:44,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:44,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c070cec3d73288b0d08a8b HTTP/1.1" 200 None 2026-03-24 22:44:44,683 - lava_callback - DEBUG - Processing node 69c070cec3d73288b0d08a8b: baseline-arm-baylibre 2026-03-24 22:44:44,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c070cec3d73288b0d08a8b 2026-03-24 22:44:44,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:44,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c070cec3d73288b0d08a8b HTTP/1.1" 200 None 2026-03-24 22:44:44,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:44,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:44:44,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:44:44,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c070cec3d73288b0d08a8b HTTP/1.1" 200 None 2026-03-24 22:44:44,736 - lava_callback - ERROR - Error processing callback for node 69c070cec3d73288b0d08a8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:45:31,186 - lava_callback - INFO - Received callback for node 69c070f6c3d73288b0d08a9c INFO: 172.21.0.1:39280 - "POST /node/69c070f6c3d73288b0d08a9c HTTP/1.0" 202 Accepted 2026-03-24 22:45:31,191 - lava_callback - INFO - Starting async job submit for node 69c070f6c3d73288b0d08a9c 2026-03-24 22:45:31,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:31,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c070f6c3d73288b0d08a9c HTTP/1.1" 200 None 2026-03-24 22:45:31,225 - lava_callback - DEBUG - Processing node 69c070f6c3d73288b0d08a9c: baseline-arm-clabbe 2026-03-24 22:45:31,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c070f6c3d73288b0d08a9c 2026-03-24 22:45:31,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:31,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c070f6c3d73288b0d08a9c HTTP/1.1" 200 None 2026-03-24 22:45:31,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:31,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:45:31,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:31,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c070f6c3d73288b0d08a9c HTTP/1.1" 200 None 2026-03-24 22:45:31,306 - lava_callback - ERROR - Error processing callback for node 69c070f6c3d73288b0d08a9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44812 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:45:51,087 - lava_callback - INFO - Received callback for node 69c0711ac3d73288b0d08aa7 INFO: 172.21.0.1:39260 - "POST /node/69c0711ac3d73288b0d08aa7 HTTP/1.0" 202 Accepted 2026-03-24 22:45:51,093 - lava_callback - INFO - Starting async job submit for node 69c0711ac3d73288b0d08aa7 2026-03-24 22:45:51,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:51,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0711ac3d73288b0d08aa7 HTTP/1.1" 200 None 2026-03-24 22:45:51,111 - lava_callback - DEBUG - Processing node 69c0711ac3d73288b0d08aa7: baseline-arm-clabbe 2026-03-24 22:45:51,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0711ac3d73288b0d08aa7 2026-03-24 22:45:51,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:51,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0711ac3d73288b0d08aa7 HTTP/1.1" 200 None 2026-03-24 22:45:51,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:51,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:45:51,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:45:51,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0711ac3d73288b0d08aa7 HTTP/1.1" 200 None 2026-03-24 22:45:51,171 - lava_callback - ERROR - Error processing callback for node 69c0711ac3d73288b0d08aa7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52012 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:46:24,794 - lava_callback - INFO - Received callback for node 69c0713ac3d73288b0d08ae0 INFO: 172.21.0.1:49876 - "POST /node/69c0713ac3d73288b0d08ae0 HTTP/1.0" 202 Accepted 2026-03-24 22:46:24,799 - lava_callback - INFO - Starting async job submit for node 69c0713ac3d73288b0d08ae0 2026-03-24 22:46:24,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:46:24,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0713ac3d73288b0d08ae0 HTTP/1.1" 200 None 2026-03-24 22:46:24,815 - lava_callback - DEBUG - Processing node 69c0713ac3d73288b0d08ae0: baseline-arm-baylibre 2026-03-24 22:46:24,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0713ac3d73288b0d08ae0 2026-03-24 22:46:24,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:46:24,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0713ac3d73288b0d08ae0 HTTP/1.1" 200 None 2026-03-24 22:46:24,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:46:24,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:46:24,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:46:24,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0713ac3d73288b0d08ae0 HTTP/1.1" 200 None 2026-03-24 22:46:24,882 - lava_callback - ERROR - Error processing callback for node 69c0713ac3d73288b0d08ae0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57192 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:47:04,813 - lava_callback - INFO - Received callback for node 69c0715dc3d73288b0d08aef INFO: 172.21.0.1:57250 - "POST /node/69c0715dc3d73288b0d08aef HTTP/1.0" 202 Accepted 2026-03-24 22:47:04,819 - lava_callback - INFO - Starting async job submit for node 69c0715dc3d73288b0d08aef 2026-03-24 22:47:04,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:47:04,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0715dc3d73288b0d08aef HTTP/1.1" 200 None 2026-03-24 22:47:04,835 - lava_callback - DEBUG - Processing node 69c0715dc3d73288b0d08aef: baseline-arm-baylibre 2026-03-24 22:47:04,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0715dc3d73288b0d08aef 2026-03-24 22:47:04,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:47:04,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0715dc3d73288b0d08aef HTTP/1.1" 200 None 2026-03-24 22:47:04,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:47:04,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:47:04,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:47:04,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0715dc3d73288b0d08aef HTTP/1.1" 200 None 2026-03-24 22:47:04,894 - lava_callback - ERROR - Error processing callback for node 69c0715dc3d73288b0d08aef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53508 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:48:44,934 - lava_callback - INFO - Received callback for node 69c071c3c3d73288b0d08b16 INFO: 172.21.0.1:51226 - "POST /node/69c071c3c3d73288b0d08b16 HTTP/1.0" 202 Accepted 2026-03-24 22:48:44,940 - lava_callback - INFO - Starting async job submit for node 69c071c3c3d73288b0d08b16 2026-03-24 22:48:44,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:48:44,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c071c3c3d73288b0d08b16 HTTP/1.1" 200 None 2026-03-24 22:48:44,956 - lava_callback - DEBUG - Processing node 69c071c3c3d73288b0d08b16: baseline-arm-baylibre 2026-03-24 22:48:44,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c071c3c3d73288b0d08b16 2026-03-24 22:48:44,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:48:44,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c071c3c3d73288b0d08b16 HTTP/1.1" 200 None 2026-03-24 22:48:44,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:48:44,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:48:44,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:48:45,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c071c3c3d73288b0d08b16 HTTP/1.1" 200 None 2026-03-24 22:48:45,013 - lava_callback - ERROR - Error processing callback for node 69c071c3c3d73288b0d08b16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35524 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:49:25,013 - lava_callback - INFO - Received callback for node 69c071e7c3d73288b0d08b1f INFO: 172.21.0.1:38846 - "POST /node/69c071e7c3d73288b0d08b1f HTTP/1.0" 202 Accepted 2026-03-24 22:49:25,018 - lava_callback - INFO - Starting async job submit for node 69c071e7c3d73288b0d08b1f 2026-03-24 22:49:25,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:49:25,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c071e7c3d73288b0d08b1f HTTP/1.1" 200 None 2026-03-24 22:49:25,034 - lava_callback - DEBUG - Processing node 69c071e7c3d73288b0d08b1f: baseline-arm-baylibre 2026-03-24 22:49:25,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c071e7c3d73288b0d08b1f 2026-03-24 22:49:25,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:49:25,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c071e7c3d73288b0d08b1f HTTP/1.1" 200 None 2026-03-24 22:49:25,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:49:25,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:49:25,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:49:25,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c071e7c3d73288b0d08b1f HTTP/1.1" 200 None 2026-03-24 22:49:25,097 - lava_callback - ERROR - Error processing callback for node 69c071e7c3d73288b0d08b1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34208 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:50:05,039 - lava_callback - INFO - Received callback for node 69c0720ac3d73288b0d08b29 INFO: 172.21.0.1:44004 - "POST /node/69c0720ac3d73288b0d08b29 HTTP/1.0" 202 Accepted 2026-03-24 22:50:05,045 - lava_callback - INFO - Starting async job submit for node 69c0720ac3d73288b0d08b29 2026-03-24 22:50:05,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:05,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0720ac3d73288b0d08b29 HTTP/1.1" 200 None 2026-03-24 22:50:05,061 - lava_callback - DEBUG - Processing node 69c0720ac3d73288b0d08b29: baseline-arm-baylibre 2026-03-24 22:50:05,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0720ac3d73288b0d08b29 2026-03-24 22:50:05,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:05,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0720ac3d73288b0d08b29 HTTP/1.1" 200 None 2026-03-24 22:50:05,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:05,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:50:05,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:05,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0720ac3d73288b0d08b29 HTTP/1.1" 200 None 2026-03-24 22:50:05,121 - lava_callback - ERROR - Error processing callback for node 69c0720ac3d73288b0d08b29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42696 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:50:24,958 - lava_callback - INFO - Received callback for node 69c0722ec3d73288b0d08b34 INFO: 172.21.0.1:33648 - "POST /node/69c0722ec3d73288b0d08b34 HTTP/1.0" 202 Accepted 2026-03-24 22:50:24,963 - lava_callback - INFO - Starting async job submit for node 69c0722ec3d73288b0d08b34 2026-03-24 22:50:24,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:24,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0722ec3d73288b0d08b34 HTTP/1.1" 200 None 2026-03-24 22:50:24,980 - lava_callback - DEBUG - Processing node 69c0722ec3d73288b0d08b34: baseline-arm-baylibre 2026-03-24 22:50:24,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0722ec3d73288b0d08b34 2026-03-24 22:50:24,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:24,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0722ec3d73288b0d08b34 HTTP/1.1" 200 None 2026-03-24 22:50:24,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:25,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:50:25,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:50:25,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0722ec3d73288b0d08b34 HTTP/1.1" 200 None 2026-03-24 22:50:25,065 - lava_callback - ERROR - Error processing callback for node 69c0722ec3d73288b0d08b34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47108 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:51:05,017 - lava_callback - INFO - Received callback for node 69c07253c3d73288b0d08b40 INFO: 172.21.0.1:49662 - "POST /node/69c07253c3d73288b0d08b40 HTTP/1.0" 202 Accepted 2026-03-24 22:51:05,022 - lava_callback - INFO - Starting async job submit for node 69c07253c3d73288b0d08b40 2026-03-24 22:51:05,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:05,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07253c3d73288b0d08b40 HTTP/1.1" 200 None 2026-03-24 22:51:05,039 - lava_callback - DEBUG - Processing node 69c07253c3d73288b0d08b40: baseline-arm-baylibre 2026-03-24 22:51:05,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07253c3d73288b0d08b40 2026-03-24 22:51:05,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:05,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07253c3d73288b0d08b40 HTTP/1.1" 200 None 2026-03-24 22:51:05,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:05,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:51:05,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:05,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07253c3d73288b0d08b40 HTTP/1.1" 200 None 2026-03-24 22:51:05,104 - lava_callback - ERROR - Error processing callback for node 69c07253c3d73288b0d08b40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39446 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:51:45,112 - lava_callback - INFO - Received callback for node 69c07276c3d73288b0d08b4c INFO: 172.21.0.1:48056 - "POST /node/69c07276c3d73288b0d08b4c HTTP/1.0" 202 Accepted 2026-03-24 22:51:45,121 - lava_callback - INFO - Starting async job submit for node 69c07276c3d73288b0d08b4c 2026-03-24 22:51:45,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:45,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07276c3d73288b0d08b4c HTTP/1.1" 200 None 2026-03-24 22:51:45,139 - lava_callback - DEBUG - Processing node 69c07276c3d73288b0d08b4c: baseline-arm-baylibre 2026-03-24 22:51:45,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07276c3d73288b0d08b4c 2026-03-24 22:51:45,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:45,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07276c3d73288b0d08b4c HTTP/1.1" 200 None 2026-03-24 22:51:45,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:45,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:51:45,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:45,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07276c3d73288b0d08b4c HTTP/1.1" 200 None 2026-03-24 22:51:45,204 - lava_callback - ERROR - Error processing callback for node 69c07276c3d73288b0d08b4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48068 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:51:54,982 - lava_callback - INFO - Received callback for node 69c307306778148209fd6a26 INFO: 172.21.0.1:37420 - "POST /node/69c307306778148209fd6a26 HTTP/1.0" 202 Accepted 2026-03-24 22:51:55,033 - lava_callback - INFO - Starting async job submit for node 69c307306778148209fd6a26 2026-03-24 22:51:55,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:55,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c307306778148209fd6a26 HTTP/1.1" 200 None 2026-03-24 22:51:55,073 - lava_callback - DEBUG - Processing node 69c307306778148209fd6a26: baseline-arm-pengutronix 2026-03-24 22:51:55,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 22:51:55,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 22:51:55,376 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c307306778148209fd6a26/log.txt.gz 2026-03-24 22:51:55,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 22:51:55,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 22:51:55,622 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c307306778148209fd6a26/lava-logs.yaml 2026-03-24 22:51:55,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 22:51:55,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 22:51:55,768 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c307306778148209fd6a26/lava_callback.json.gz DEBUG: Setup failed for 69c307306778148209fd6a26. Transitting job node result: incomplete -> fail 2026-03-24 22:51:55,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:55,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c307306778148209fd6a26 HTTP/1.1" 200 None 2026-03-24 22:51:55,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:55,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-24 22:51:55,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:51:55,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c307306778148209fd6a26 HTTP/1.1" 200 None 2026-03-24 22:51:55,932 - lava_callback - INFO - Completed processing callback for node 69c307306778148209fd6a26 INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:52:05,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:52:05,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41938 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:52:31,328 - lava_callback - INFO - Received callback for node 69c0729cc3d73288b0d08b57 INFO: 172.21.0.1:32932 - "POST /node/69c0729cc3d73288b0d08b57 HTTP/1.0" 202 Accepted 2026-03-24 22:52:31,333 - lava_callback - INFO - Starting async job submit for node 69c0729cc3d73288b0d08b57 2026-03-24 22:52:31,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:52:31,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0729cc3d73288b0d08b57 HTTP/1.1" 200 None 2026-03-24 22:52:31,350 - lava_callback - DEBUG - Processing node 69c0729cc3d73288b0d08b57: baseline-arm-clabbe 2026-03-24 22:52:31,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0729cc3d73288b0d08b57 2026-03-24 22:52:31,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:52:31,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0729cc3d73288b0d08b57 HTTP/1.1" 200 None 2026-03-24 22:52:31,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:52:31,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:52:31,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:52:31,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0729cc3d73288b0d08b57 HTTP/1.1" 200 None 2026-03-24 22:52:31,405 - lava_callback - ERROR - Error processing callback for node 69c0729cc3d73288b0d08b57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33988 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:54:11,470 - lava_callback - INFO - Received callback for node 69c07304c3d73288b0d08b76 INFO: 172.21.0.1:33932 - "POST /node/69c07304c3d73288b0d08b76 HTTP/1.0" 202 Accepted 2026-03-24 22:54:11,476 - lava_callback - INFO - Starting async job submit for node 69c07304c3d73288b0d08b76 2026-03-24 22:54:11,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:11,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07304c3d73288b0d08b76 HTTP/1.1" 200 None 2026-03-24 22:54:11,491 - lava_callback - DEBUG - Processing node 69c07304c3d73288b0d08b76: baseline-arm-clabbe 2026-03-24 22:54:11,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07304c3d73288b0d08b76 2026-03-24 22:54:11,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:11,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07304c3d73288b0d08b76 HTTP/1.1" 200 None 2026-03-24 22:54:11,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:11,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:54:11,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:11,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07304c3d73288b0d08b76 HTTP/1.1" 200 None 2026-03-24 22:54:11,573 - lava_callback - ERROR - Error processing callback for node 69c07304c3d73288b0d08b76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45286 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:54:51,426 - lava_callback - INFO - Received callback for node 69c07329c3d73288b0d08b83 INFO: 172.21.0.1:52610 - "POST /node/69c07329c3d73288b0d08b83 HTTP/1.0" 202 Accepted 2026-03-24 22:54:51,432 - lava_callback - INFO - Starting async job submit for node 69c07329c3d73288b0d08b83 2026-03-24 22:54:51,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:51,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07329c3d73288b0d08b83 HTTP/1.1" 200 None 2026-03-24 22:54:51,448 - lava_callback - DEBUG - Processing node 69c07329c3d73288b0d08b83: baseline-arm-clabbe 2026-03-24 22:54:51,448 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07329c3d73288b0d08b83 2026-03-24 22:54:51,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:51,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07329c3d73288b0d08b83 HTTP/1.1" 200 None 2026-03-24 22:54:51,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:51,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:54:51,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:54:51,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07329c3d73288b0d08b83 HTTP/1.1" 200 None 2026-03-24 22:54:51,515 - lava_callback - ERROR - Error processing callback for node 69c07329c3d73288b0d08b83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49018 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:55:31,456 - lava_callback - INFO - Received callback for node 69c0734dc3d73288b0d08b92 INFO: 172.21.0.1:45824 - "POST /node/69c0734dc3d73288b0d08b92 HTTP/1.0" 202 Accepted 2026-03-24 22:55:31,462 - lava_callback - INFO - Starting async job submit for node 69c0734dc3d73288b0d08b92 2026-03-24 22:55:31,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:31,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0734dc3d73288b0d08b92 HTTP/1.1" 200 None 2026-03-24 22:55:31,478 - lava_callback - DEBUG - Processing node 69c0734dc3d73288b0d08b92: baseline-arm-clabbe 2026-03-24 22:55:31,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0734dc3d73288b0d08b92 2026-03-24 22:55:31,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:31,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0734dc3d73288b0d08b92 HTTP/1.1" 200 None 2026-03-24 22:55:31,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:31,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:55:31,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:31,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0734dc3d73288b0d08b92 HTTP/1.1" 200 None 2026-03-24 22:55:31,533 - lava_callback - ERROR - Error processing callback for node 69c0734dc3d73288b0d08b92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35332 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:55:51,465 - lava_callback - INFO - Received callback for node 69c07371c3d73288b0d08b9f INFO: 172.21.0.1:42004 - "POST /node/69c07371c3d73288b0d08b9f HTTP/1.0" 202 Accepted 2026-03-24 22:55:51,471 - lava_callback - INFO - Starting async job submit for node 69c07371c3d73288b0d08b9f 2026-03-24 22:55:51,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:51,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07371c3d73288b0d08b9f HTTP/1.1" 200 None 2026-03-24 22:55:51,486 - lava_callback - DEBUG - Processing node 69c07371c3d73288b0d08b9f: baseline-arm-clabbe 2026-03-24 22:55:51,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07371c3d73288b0d08b9f 2026-03-24 22:55:51,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:51,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07371c3d73288b0d08b9f HTTP/1.1" 200 None 2026-03-24 22:55:51,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:51,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:55:51,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:55:51,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07371c3d73288b0d08b9f HTTP/1.1" 200 None 2026-03-24 22:55:51,545 - lava_callback - ERROR - Error processing callback for node 69c07371c3d73288b0d08b9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52354 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:56:31,546 - lava_callback - INFO - Received callback for node 69c07395c3d73288b0d08bb0 INFO: 172.21.0.1:57306 - "POST /node/69c07395c3d73288b0d08bb0 HTTP/1.0" 202 Accepted 2026-03-24 22:56:31,552 - lava_callback - INFO - Starting async job submit for node 69c07395c3d73288b0d08bb0 2026-03-24 22:56:31,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:56:31,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07395c3d73288b0d08bb0 HTTP/1.1" 200 None 2026-03-24 22:56:31,568 - lava_callback - DEBUG - Processing node 69c07395c3d73288b0d08bb0: baseline-arm-clabbe 2026-03-24 22:56:31,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07395c3d73288b0d08bb0 2026-03-24 22:56:31,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:56:31,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07395c3d73288b0d08bb0 HTTP/1.1" 200 None 2026-03-24 22:56:31,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:56:31,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-24 22:56:31,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:56:31,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07395c3d73288b0d08bb0 HTTP/1.1" 200 None 2026-03-24 22:56:31,629 - lava_callback - ERROR - Error processing callback for node 69c07395c3d73288b0d08bb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59320 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:57:05,761 - lava_callback - INFO - Received callback for node 69c073b4c3d73288b0d08bb9 INFO: 172.21.0.1:53108 - "POST /node/69c073b4c3d73288b0d08bb9 HTTP/1.0" 202 Accepted 2026-03-24 22:57:05,767 - lava_callback - INFO - Starting async job submit for node 69c073b4c3d73288b0d08bb9 2026-03-24 22:57:05,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:05,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c073b4c3d73288b0d08bb9 HTTP/1.1" 200 None 2026-03-24 22:57:05,784 - lava_callback - DEBUG - Processing node 69c073b4c3d73288b0d08bb9: baseline-arm-baylibre 2026-03-24 22:57:05,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c073b4c3d73288b0d08bb9 2026-03-24 22:57:05,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:05,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c073b4c3d73288b0d08bb9 HTTP/1.1" 200 None 2026-03-24 22:57:05,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:05,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:57:05,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:05,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c073b4c3d73288b0d08bb9 HTTP/1.1" 200 None 2026-03-24 22:57:05,872 - lava_callback - ERROR - Error processing callback for node 69c073b4c3d73288b0d08bb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40384 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:57:51,587 - lava_callback - INFO - Received callback for node 69c073dcc3d73288b0d08bc7 INFO: 172.21.0.1:55860 - "POST /node/69c073dcc3d73288b0d08bc7 HTTP/1.0" 202 Accepted 2026-03-24 22:57:51,593 - lava_callback - INFO - Starting async job submit for node 69c073dcc3d73288b0d08bc7 2026-03-24 22:57:51,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:51,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c073dcc3d73288b0d08bc7 HTTP/1.1" 200 None 2026-03-24 22:57:51,613 - lava_callback - DEBUG - Processing node 69c073dcc3d73288b0d08bc7: baseline-arm-clabbe 2026-03-24 22:57:51,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c073dcc3d73288b0d08bc7 2026-03-24 22:57:51,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:51,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c073dcc3d73288b0d08bc7 HTTP/1.1" 200 None 2026-03-24 22:57:51,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:51,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:57:51,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:57:51,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c073dcc3d73288b0d08bc7 HTTP/1.1" 200 None 2026-03-24 22:57:51,670 - lava_callback - ERROR - Error processing callback for node 69c073dcc3d73288b0d08bc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59138 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:58:11,584 - lava_callback - INFO - Received callback for node 69c07400c3d73288b0d08bd4 INFO: 172.21.0.1:51734 - "POST /node/69c07400c3d73288b0d08bd4 HTTP/1.0" 202 Accepted 2026-03-24 22:58:11,590 - lava_callback - INFO - Starting async job submit for node 69c07400c3d73288b0d08bd4 2026-03-24 22:58:11,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:11,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07400c3d73288b0d08bd4 HTTP/1.1" 200 None 2026-03-24 22:58:11,606 - lava_callback - DEBUG - Processing node 69c07400c3d73288b0d08bd4: baseline-arm-clabbe 2026-03-24 22:58:11,606 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07400c3d73288b0d08bd4 2026-03-24 22:58:11,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:11,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07400c3d73288b0d08bd4 HTTP/1.1" 200 None 2026-03-24 22:58:11,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:11,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:58:11,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:11,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07400c3d73288b0d08bd4 HTTP/1.1" 200 None 2026-03-24 22:58:11,670 - lava_callback - ERROR - Error processing callback for node 69c07400c3d73288b0d08bd4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57750 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:58:45,451 - lava_callback - INFO - Received callback for node 69c07421c3d73288b0d08bdd INFO: 172.21.0.1:35864 - "POST /node/69c07421c3d73288b0d08bdd HTTP/1.0" 202 Accepted 2026-03-24 22:58:45,457 - lava_callback - INFO - Starting async job submit for node 69c07421c3d73288b0d08bdd 2026-03-24 22:58:45,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:45,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07421c3d73288b0d08bdd HTTP/1.1" 200 None 2026-03-24 22:58:45,472 - lava_callback - DEBUG - Processing node 69c07421c3d73288b0d08bdd: baseline-arm-baylibre 2026-03-24 22:58:45,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07421c3d73288b0d08bdd 2026-03-24 22:58:45,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:45,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07421c3d73288b0d08bdd HTTP/1.1" 200 None 2026-03-24 22:58:45,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:45,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:58:45,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:58:45,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07421c3d73288b0d08bdd HTTP/1.1" 200 None 2026-03-24 22:58:45,524 - lava_callback - ERROR - Error processing callback for node 69c07421c3d73288b0d08bdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41456 - "GET / HTTP/1.0" 200 OK 2026-03-24 22:59:25,446 - lava_callback - INFO - Received callback for node 69c07442c3d73288b0d08bea INFO: 172.21.0.1:57964 - "POST /node/69c07442c3d73288b0d08bea HTTP/1.0" 202 Accepted 2026-03-24 22:59:25,451 - lava_callback - INFO - Starting async job submit for node 69c07442c3d73288b0d08bea 2026-03-24 22:59:25,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:59:25,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07442c3d73288b0d08bea HTTP/1.1" 200 None 2026-03-24 22:59:25,468 - lava_callback - DEBUG - Processing node 69c07442c3d73288b0d08bea: baseline-arm-baylibre 2026-03-24 22:59:25,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07442c3d73288b0d08bea 2026-03-24 22:59:25,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:59:25,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07442c3d73288b0d08bea HTTP/1.1" 200 None 2026-03-24 22:59:25,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:59:25,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 22:59:25,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 22:59:25,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07442c3d73288b0d08bea HTTP/1.1" 200 None 2026-03-24 22:59:25,520 - lava_callback - ERROR - Error processing callback for node 69c07442c3d73288b0d08bea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46756 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:00:05,508 - lava_callback - INFO - Received callback for node 69c07466c3d73288b0d08bf3 INFO: 172.21.0.1:43768 - "POST /node/69c07466c3d73288b0d08bf3 HTTP/1.0" 202 Accepted 2026-03-24 23:00:05,513 - lava_callback - INFO - Starting async job submit for node 69c07466c3d73288b0d08bf3 2026-03-24 23:00:05,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:05,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07466c3d73288b0d08bf3 HTTP/1.1" 200 None 2026-03-24 23:00:05,529 - lava_callback - DEBUG - Processing node 69c07466c3d73288b0d08bf3: baseline-arm-baylibre 2026-03-24 23:00:05,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07466c3d73288b0d08bf3 2026-03-24 23:00:05,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:05,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07466c3d73288b0d08bf3 HTTP/1.1" 200 None 2026-03-24 23:00:05,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:05,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:00:05,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:05,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07466c3d73288b0d08bf3 HTTP/1.1" 200 None 2026-03-24 23:00:05,580 - lava_callback - ERROR - Error processing callback for node 69c07466c3d73288b0d08bf3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55520 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:00:45,528 - lava_callback - INFO - Received callback for node 69c0748bc3d73288b0d08bfc INFO: 172.21.0.1:45456 - "POST /node/69c0748bc3d73288b0d08bfc HTTP/1.0" 202 Accepted 2026-03-24 23:00:45,533 - lava_callback - INFO - Starting async job submit for node 69c0748bc3d73288b0d08bfc 2026-03-24 23:00:45,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:45,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0748bc3d73288b0d08bfc HTTP/1.1" 200 None 2026-03-24 23:00:45,558 - lava_callback - DEBUG - Processing node 69c0748bc3d73288b0d08bfc: baseline-arm-baylibre 2026-03-24 23:00:45,558 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0748bc3d73288b0d08bfc 2026-03-24 23:00:45,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:45,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0748bc3d73288b0d08bfc HTTP/1.1" 200 None 2026-03-24 23:00:45,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:45,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:00:45,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:00:45,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0748bc3d73288b0d08bfc HTTP/1.1" 200 None 2026-03-24 23:00:45,621 - lava_callback - ERROR - Error processing callback for node 69c0748bc3d73288b0d08bfc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45960 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:01:05,544 - lava_callback - INFO - Received callback for node 69c074adc3d73288b0d08c09 INFO: 172.21.0.1:43194 - "POST /node/69c074adc3d73288b0d08c09 HTTP/1.0" 202 Accepted 2026-03-24 23:01:05,550 - lava_callback - INFO - Starting async job submit for node 69c074adc3d73288b0d08c09 2026-03-24 23:01:05,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:05,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c074adc3d73288b0d08c09 HTTP/1.1" 200 None 2026-03-24 23:01:05,566 - lava_callback - DEBUG - Processing node 69c074adc3d73288b0d08c09: baseline-arm-baylibre 2026-03-24 23:01:05,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c074adc3d73288b0d08c09 2026-03-24 23:01:05,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:05,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c074adc3d73288b0d08c09 HTTP/1.1" 200 None 2026-03-24 23:01:05,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:05,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:01:05,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:05,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c074adc3d73288b0d08c09 HTTP/1.1" 200 None 2026-03-24 23:01:05,618 - lava_callback - ERROR - Error processing callback for node 69c074adc3d73288b0d08c09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33250 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:01:45,636 - lava_callback - INFO - Received callback for node 69c074d1c3d73288b0d08c14 INFO: 172.21.0.1:45486 - "POST /node/69c074d1c3d73288b0d08c14 HTTP/1.0" 202 Accepted 2026-03-24 23:01:45,642 - lava_callback - INFO - Starting async job submit for node 69c074d1c3d73288b0d08c14 2026-03-24 23:01:45,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:45,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c074d1c3d73288b0d08c14 HTTP/1.1" 200 None 2026-03-24 23:01:45,668 - lava_callback - DEBUG - Processing node 69c074d1c3d73288b0d08c14: baseline-arm-baylibre 2026-03-24 23:01:45,669 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c074d1c3d73288b0d08c14 2026-03-24 23:01:45,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:45,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c074d1c3d73288b0d08c14 HTTP/1.1" 200 None 2026-03-24 23:01:45,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:45,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:01:45,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:01:45,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c074d1c3d73288b0d08c14 HTTP/1.1" 200 None 2026-03-24 23:01:45,735 - lava_callback - ERROR - Error processing callback for node 69c074d1c3d73288b0d08c14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:02:31,840 - lava_callback - INFO - Received callback for node 69c074f7c3d73288b0d08c25 INFO: 172.21.0.1:56022 - "POST /node/69c074f7c3d73288b0d08c25 HTTP/1.0" 202 Accepted 2026-03-24 23:02:31,846 - lava_callback - INFO - Starting async job submit for node 69c074f7c3d73288b0d08c25 2026-03-24 23:02:31,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:02:31,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c074f7c3d73288b0d08c25 HTTP/1.1" 200 None 2026-03-24 23:02:31,862 - lava_callback - DEBUG - Processing node 69c074f7c3d73288b0d08c25: baseline-arm-clabbe 2026-03-24 23:02:31,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c074f7c3d73288b0d08c25 2026-03-24 23:02:31,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:02:31,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c074f7c3d73288b0d08c25 HTTP/1.1" 200 None 2026-03-24 23:02:31,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:02:31,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:02:31,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:02:31,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c074f7c3d73288b0d08c25 HTTP/1.1" 200 None 2026-03-24 23:02:31,919 - lava_callback - ERROR - Error processing callback for node 69c074f7c3d73288b0d08c25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53214 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:03:05,699 - lava_callback - INFO - Received callback for node 69c0751ac3d73288b0d08c2d INFO: 172.21.0.1:45086 - "POST /node/69c0751ac3d73288b0d08c2d HTTP/1.0" 202 Accepted 2026-03-24 23:03:05,705 - lava_callback - INFO - Starting async job submit for node 69c0751ac3d73288b0d08c2d 2026-03-24 23:03:05,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:05,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0751ac3d73288b0d08c2d HTTP/1.1" 200 None 2026-03-24 23:03:05,726 - lava_callback - DEBUG - Processing node 69c0751ac3d73288b0d08c2d: baseline-arm-baylibre 2026-03-24 23:03:05,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0751ac3d73288b0d08c2d 2026-03-24 23:03:05,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:05,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0751ac3d73288b0d08c2d HTTP/1.1" 200 None 2026-03-24 23:03:05,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:05,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:03:05,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:05,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0751ac3d73288b0d08c2d HTTP/1.1" 200 None 2026-03-24 23:03:05,807 - lava_callback - ERROR - Error processing callback for node 69c0751ac3d73288b0d08c2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55244 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:03:51,855 - lava_callback - INFO - Received callback for node 69c07542c3d73288b0d08c38 INFO: 172.21.0.1:50236 - "POST /node/69c07542c3d73288b0d08c38 HTTP/1.0" 202 Accepted 2026-03-24 23:03:51,861 - lava_callback - INFO - Starting async job submit for node 69c07542c3d73288b0d08c38 2026-03-24 23:03:51,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:51,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07542c3d73288b0d08c38 HTTP/1.1" 200 None 2026-03-24 23:03:51,877 - lava_callback - DEBUG - Processing node 69c07542c3d73288b0d08c38: baseline-arm-clabbe 2026-03-24 23:03:51,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07542c3d73288b0d08c38 2026-03-24 23:03:51,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:51,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07542c3d73288b0d08c38 HTTP/1.1" 200 None 2026-03-24 23:03:51,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:51,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:03:51,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:03:51,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07542c3d73288b0d08c38 HTTP/1.1" 200 None 2026-03-24 23:03:51,929 - lava_callback - ERROR - Error processing callback for node 69c07542c3d73288b0d08c38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40338 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:04:11,825 - lava_callback - INFO - Received callback for node 69c07569c3d73288b0d08c47 INFO: 172.21.0.1:33574 - "POST /node/69c07569c3d73288b0d08c47 HTTP/1.0" 202 Accepted 2026-03-24 23:04:11,831 - lava_callback - INFO - Starting async job submit for node 69c07569c3d73288b0d08c47 2026-03-24 23:04:11,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:04:11,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07569c3d73288b0d08c47 HTTP/1.1" 200 None 2026-03-24 23:04:11,847 - lava_callback - DEBUG - Processing node 69c07569c3d73288b0d08c47: baseline-arm-clabbe 2026-03-24 23:04:11,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07569c3d73288b0d08c47 2026-03-24 23:04:11,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:04:11,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07569c3d73288b0d08c47 HTTP/1.1" 200 None 2026-03-24 23:04:11,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:04:11,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:04:11,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:04:11,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07569c3d73288b0d08c47 HTTP/1.1" 200 None 2026-03-24 23:04:11,902 - lava_callback - ERROR - Error processing callback for node 69c07569c3d73288b0d08c47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56704 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:05:05,928 - lava_callback - INFO - Received callback for node 69c0758fc3d73288b0d08c53 INFO: 172.21.0.1:47784 - "POST /node/69c0758fc3d73288b0d08c53 HTTP/1.0" 202 Accepted 2026-03-24 23:05:05,934 - lava_callback - INFO - Starting async job submit for node 69c0758fc3d73288b0d08c53 2026-03-24 23:05:05,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:05,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0758fc3d73288b0d08c53 HTTP/1.1" 200 None 2026-03-24 23:05:05,949 - lava_callback - DEBUG - Processing node 69c0758fc3d73288b0d08c53: baseline-arm-baylibre 2026-03-24 23:05:05,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0758fc3d73288b0d08c53 2026-03-24 23:05:05,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:05,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0758fc3d73288b0d08c53 HTTP/1.1" 200 None 2026-03-24 23:05:05,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:05,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:05:05,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:06,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0758fc3d73288b0d08c53 HTTP/1.1" 200 None 2026-03-24 23:05:06,005 - lava_callback - ERROR - Error processing callback for node 69c0758fc3d73288b0d08c53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60680 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:05:31,980 - lava_callback - INFO - Received callback for node 69c075b6c3d73288b0d08c64 INFO: 172.21.0.1:49282 - "POST /node/69c075b6c3d73288b0d08c64 HTTP/1.0" 202 Accepted 2026-03-24 23:05:31,986 - lava_callback - INFO - Starting async job submit for node 69c075b6c3d73288b0d08c64 2026-03-24 23:05:31,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:32,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c075b6c3d73288b0d08c64 HTTP/1.1" 200 None 2026-03-24 23:05:32,003 - lava_callback - DEBUG - Processing node 69c075b6c3d73288b0d08c64: baseline-arm-clabbe 2026-03-24 23:05:32,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c075b6c3d73288b0d08c64 2026-03-24 23:05:32,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:32,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c075b6c3d73288b0d08c64 HTTP/1.1" 200 None 2026-03-24 23:05:32,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:32,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:05:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:05:32,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c075b6c3d73288b0d08c64 HTTP/1.1" 200 None 2026-03-24 23:05:32,057 - lava_callback - ERROR - Error processing callback for node 69c075b6c3d73288b0d08c64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57986 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:06:12,983 - lava_callback - INFO - Received callback for node 69c075dcc3d73288b0d08c71 INFO: 172.21.0.1:46946 - "POST /node/69c075dcc3d73288b0d08c71 HTTP/1.0" 202 Accepted 2026-03-24 23:06:12,989 - lava_callback - INFO - Starting async job submit for node 69c075dcc3d73288b0d08c71 2026-03-24 23:06:12,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:13,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c075dcc3d73288b0d08c71 HTTP/1.1" 200 None 2026-03-24 23:06:13,015 - lava_callback - DEBUG - Processing node 69c075dcc3d73288b0d08c71: baseline-arm-clabbe 2026-03-24 23:06:13,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c075dcc3d73288b0d08c71 2026-03-24 23:06:13,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:13,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c075dcc3d73288b0d08c71 HTTP/1.1" 200 None 2026-03-24 23:06:13,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:13,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:06:13,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:13,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c075dcc3d73288b0d08c71 HTTP/1.1" 200 None 2026-03-24 23:06:13,100 - lava_callback - ERROR - Error processing callback for node 69c075dcc3d73288b0d08c71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44166 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:06:51,964 - lava_callback - INFO - Received callback for node 69c07607c3d73288b0d08c7c INFO: 172.21.0.1:47968 - "POST /node/69c07607c3d73288b0d08c7c HTTP/1.0" 202 Accepted 2026-03-24 23:06:51,970 - lava_callback - INFO - Starting async job submit for node 69c07607c3d73288b0d08c7c 2026-03-24 23:06:51,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:51,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07607c3d73288b0d08c7c HTTP/1.1" 200 None 2026-03-24 23:06:51,992 - lava_callback - DEBUG - Processing node 69c07607c3d73288b0d08c7c: baseline-arm-clabbe 2026-03-24 23:06:51,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07607c3d73288b0d08c7c 2026-03-24 23:06:51,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:52,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07607c3d73288b0d08c7c HTTP/1.1" 200 None 2026-03-24 23:06:52,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:52,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:06:52,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:06:52,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07607c3d73288b0d08c7c HTTP/1.1" 200 None 2026-03-24 23:06:52,056 - lava_callback - ERROR - Error processing callback for node 69c07607c3d73288b0d08c7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51646 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:07:32,188 - lava_callback - INFO - Received callback for node 69c0762fc3d73288b0d08c86 INFO: 172.21.0.1:51108 - "POST /node/69c0762fc3d73288b0d08c86 HTTP/1.0" 202 Accepted 2026-03-24 23:07:32,193 - lava_callback - INFO - Starting async job submit for node 69c0762fc3d73288b0d08c86 2026-03-24 23:07:32,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:07:32,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0762fc3d73288b0d08c86 HTTP/1.1" 200 None 2026-03-24 23:07:32,210 - lava_callback - DEBUG - Processing node 69c0762fc3d73288b0d08c86: baseline-arm-clabbe 2026-03-24 23:07:32,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0762fc3d73288b0d08c86 2026-03-24 23:07:32,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:07:32,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0762fc3d73288b0d08c86 HTTP/1.1" 200 None 2026-03-24 23:07:32,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:07:32,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:07:32,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:07:32,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0762fc3d73288b0d08c86 HTTP/1.1" 200 None 2026-03-24 23:07:32,298 - lava_callback - ERROR - Error processing callback for node 69c0762fc3d73288b0d08c86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52044 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:08:12,073 - lava_callback - INFO - Received callback for node 69c07655c3d73288b0d08c91 INFO: 172.21.0.1:43476 - "POST /node/69c07655c3d73288b0d08c91 HTTP/1.0" 202 Accepted 2026-03-24 23:08:12,079 - lava_callback - INFO - Starting async job submit for node 69c07655c3d73288b0d08c91 2026-03-24 23:08:12,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:08:12,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07655c3d73288b0d08c91 HTTP/1.1" 200 None 2026-03-24 23:08:12,095 - lava_callback - DEBUG - Processing node 69c07655c3d73288b0d08c91: baseline-arm-clabbe 2026-03-24 23:08:12,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07655c3d73288b0d08c91 2026-03-24 23:08:12,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:08:12,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07655c3d73288b0d08c91 HTTP/1.1" 200 None 2026-03-24 23:08:12,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:08:12,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:08:12,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:08:12,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07655c3d73288b0d08c91 HTTP/1.1" 200 None 2026-03-24 23:08:12,157 - lava_callback - ERROR - Error processing callback for node 69c07655c3d73288b0d08c91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:09:06,070 - lava_callback - INFO - Received callback for node 69c07683c3d73288b0d08c9d INFO: 172.21.0.1:55792 - "POST /node/69c07683c3d73288b0d08c9d HTTP/1.0" 202 Accepted 2026-03-24 23:09:06,076 - lava_callback - INFO - Starting async job submit for node 69c07683c3d73288b0d08c9d 2026-03-24 23:09:06,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:09:06,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07683c3d73288b0d08c9d HTTP/1.1" 200 None 2026-03-24 23:09:06,094 - lava_callback - DEBUG - Processing node 69c07683c3d73288b0d08c9d: baseline-arm64-baylibre 2026-03-24 23:09:06,094 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07683c3d73288b0d08c9d 2026-03-24 23:09:06,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:09:06,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07683c3d73288b0d08c9d HTTP/1.1" 200 None 2026-03-24 23:09:06,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:09:06,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:09:06,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:09:06,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07683c3d73288b0d08c9d HTTP/1.1" 200 None 2026-03-24 23:09:06,148 - lava_callback - ERROR - Error processing callback for node 69c07683c3d73288b0d08c9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34734 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:10:12,252 - lava_callback - INFO - Received callback for node 69c076c8c3d73288b0d08ca8 INFO: 172.21.0.1:53136 - "POST /node/69c076c8c3d73288b0d08ca8 HTTP/1.0" 202 Accepted 2026-03-24 23:10:12,258 - lava_callback - INFO - Starting async job submit for node 69c076c8c3d73288b0d08ca8 2026-03-24 23:10:12,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:12,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c076c8c3d73288b0d08ca8 HTTP/1.1" 200 None 2026-03-24 23:10:12,274 - lava_callback - DEBUG - Processing node 69c076c8c3d73288b0d08ca8: baseline-arm-clabbe 2026-03-24 23:10:12,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c076c8c3d73288b0d08ca8 2026-03-24 23:10:12,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:12,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c076c8c3d73288b0d08ca8 HTTP/1.1" 200 None 2026-03-24 23:10:12,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:12,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:10:12,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:12,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c076c8c3d73288b0d08ca8 HTTP/1.1" 200 None 2026-03-24 23:10:12,357 - lava_callback - ERROR - Error processing callback for node 69c076c8c3d73288b0d08ca8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:10:52,242 - lava_callback - INFO - Received callback for node 69c076efc3d73288b0d08cb2 INFO: 172.21.0.1:34816 - "POST /node/69c076efc3d73288b0d08cb2 HTTP/1.0" 202 Accepted 2026-03-24 23:10:52,248 - lava_callback - INFO - Starting async job submit for node 69c076efc3d73288b0d08cb2 2026-03-24 23:10:52,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:52,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c076efc3d73288b0d08cb2 HTTP/1.1" 200 None 2026-03-24 23:10:52,264 - lava_callback - DEBUG - Processing node 69c076efc3d73288b0d08cb2: baseline-arm-clabbe 2026-03-24 23:10:52,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c076efc3d73288b0d08cb2 2026-03-24 23:10:52,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:52,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c076efc3d73288b0d08cb2 HTTP/1.1" 200 None 2026-03-24 23:10:52,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:52,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-24 23:10:52,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:10:52,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c076efc3d73288b0d08cb2 HTTP/1.1" 200 None 2026-03-24 23:10:52,326 - lava_callback - ERROR - Error processing callback for node 69c076efc3d73288b0d08cb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43650 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:11:52,312 - lava_callback - INFO - Received callback for node 69c07724c3d73288b0d08cbd INFO: 172.21.0.1:40800 - "POST /node/69c07724c3d73288b0d08cbd HTTP/1.0" 202 Accepted 2026-03-24 23:11:52,318 - lava_callback - INFO - Starting async job submit for node 69c07724c3d73288b0d08cbd 2026-03-24 23:11:52,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:11:52,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07724c3d73288b0d08cbd HTTP/1.1" 200 None 2026-03-24 23:11:52,345 - lava_callback - DEBUG - Processing node 69c07724c3d73288b0d08cbd: baseline-arm64-clabbe 2026-03-24 23:11:52,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07724c3d73288b0d08cbd 2026-03-24 23:11:52,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:11:52,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07724c3d73288b0d08cbd HTTP/1.1" 200 None 2026-03-24 23:11:52,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:11:52,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:11:52,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:11:52,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07724c3d73288b0d08cbd HTTP/1.1" 200 None 2026-03-24 23:11:52,408 - lava_callback - ERROR - Error processing callback for node 69c07724c3d73288b0d08cbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39932 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:13:12,372 - lava_callback - INFO - Received callback for node 69c07775c3d73288b0d08cc6 INFO: 172.21.0.1:42044 - "POST /node/69c07775c3d73288b0d08cc6 HTTP/1.0" 202 Accepted 2026-03-24 23:13:12,377 - lava_callback - INFO - Starting async job submit for node 69c07775c3d73288b0d08cc6 2026-03-24 23:13:12,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:13:12,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07775c3d73288b0d08cc6 HTTP/1.1" 200 None 2026-03-24 23:13:12,393 - lava_callback - DEBUG - Processing node 69c07775c3d73288b0d08cc6: baseline-arm64-clabbe 2026-03-24 23:13:12,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07775c3d73288b0d08cc6 2026-03-24 23:13:12,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:13:12,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07775c3d73288b0d08cc6 HTTP/1.1" 200 None 2026-03-24 23:13:12,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:13:12,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:13:12,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:13:12,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07775c3d73288b0d08cc6 HTTP/1.1" 200 None 2026-03-24 23:13:12,458 - lava_callback - ERROR - Error processing callback for node 69c07775c3d73288b0d08cc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:14:32,398 - lava_callback - INFO - Received callback for node 69c077c3c3d73288b0d08ccf INFO: 172.21.0.1:39334 - "POST /node/69c077c3c3d73288b0d08ccf HTTP/1.0" 202 Accepted 2026-03-24 23:14:32,404 - lava_callback - INFO - Starting async job submit for node 69c077c3c3d73288b0d08ccf 2026-03-24 23:14:32,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:14:32,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c077c3c3d73288b0d08ccf HTTP/1.1" 200 None 2026-03-24 23:14:32,420 - lava_callback - DEBUG - Processing node 69c077c3c3d73288b0d08ccf: baseline-arm64-clabbe 2026-03-24 23:14:32,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c077c3c3d73288b0d08ccf 2026-03-24 23:14:32,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:14:32,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c077c3c3d73288b0d08ccf HTTP/1.1" 200 None 2026-03-24 23:14:32,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:14:32,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:14:32,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:14:32,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c077c3c3d73288b0d08ccf HTTP/1.1" 200 None 2026-03-24 23:14:32,511 - lava_callback - ERROR - Error processing callback for node 69c077c3c3d73288b0d08ccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36396 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:15:52,476 - lava_callback - INFO - Received callback for node 69c07812c3d73288b0d08cdd INFO: 172.21.0.1:35268 - "POST /node/69c07812c3d73288b0d08cdd HTTP/1.0" 202 Accepted 2026-03-24 23:15:52,482 - lava_callback - INFO - Starting async job submit for node 69c07812c3d73288b0d08cdd 2026-03-24 23:15:52,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:15:52,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07812c3d73288b0d08cdd HTTP/1.1" 200 None 2026-03-24 23:15:52,498 - lava_callback - DEBUG - Processing node 69c07812c3d73288b0d08cdd: baseline-arm64-clabbe 2026-03-24 23:15:52,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07812c3d73288b0d08cdd 2026-03-24 23:15:52,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:15:52,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07812c3d73288b0d08cdd HTTP/1.1" 200 None 2026-03-24 23:15:52,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:15:52,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:15:52,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:15:52,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07812c3d73288b0d08cdd HTTP/1.1" 200 None 2026-03-24 23:15:52,557 - lava_callback - ERROR - Error processing callback for node 69c07812c3d73288b0d08cdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41226 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:17:12,451 - lava_callback - INFO - Received callback for node 69c07864c3d73288b0d08ce8 INFO: 172.21.0.1:56254 - "POST /node/69c07864c3d73288b0d08ce8 HTTP/1.0" 202 Accepted 2026-03-24 23:17:12,456 - lava_callback - INFO - Starting async job submit for node 69c07864c3d73288b0d08ce8 2026-03-24 23:17:12,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:17:12,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07864c3d73288b0d08ce8 HTTP/1.1" 200 None 2026-03-24 23:17:12,480 - lava_callback - DEBUG - Processing node 69c07864c3d73288b0d08ce8: baseline-arm64-clabbe 2026-03-24 23:17:12,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07864c3d73288b0d08ce8 2026-03-24 23:17:12,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:17:12,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07864c3d73288b0d08ce8 HTTP/1.1" 200 None 2026-03-24 23:17:12,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:17:12,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:17:12,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:17:12,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07864c3d73288b0d08ce8 HTTP/1.1" 200 None 2026-03-24 23:17:12,548 - lava_callback - ERROR - Error processing callback for node 69c07864c3d73288b0d08ce8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54090 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:18:32,666 - lava_callback - INFO - Received callback for node 69c078b7c3d73288b0d08cf4 INFO: 172.21.0.1:53174 - "POST /node/69c078b7c3d73288b0d08cf4 HTTP/1.0" 202 Accepted 2026-03-24 23:18:32,672 - lava_callback - INFO - Starting async job submit for node 69c078b7c3d73288b0d08cf4 2026-03-24 23:18:32,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:18:32,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c078b7c3d73288b0d08cf4 HTTP/1.1" 200 None 2026-03-24 23:18:32,689 - lava_callback - DEBUG - Processing node 69c078b7c3d73288b0d08cf4: baseline-arm64-clabbe 2026-03-24 23:18:32,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c078b7c3d73288b0d08cf4 2026-03-24 23:18:32,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:18:32,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c078b7c3d73288b0d08cf4 HTTP/1.1" 200 None 2026-03-24 23:18:32,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:18:32,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:18:32,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:18:32,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c078b7c3d73288b0d08cf4 HTTP/1.1" 200 None 2026-03-24 23:18:32,748 - lava_callback - ERROR - Error processing callback for node 69c078b7c3d73288b0d08cf4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43278 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:19:46,707 - lava_callback - INFO - Received callback for node 69c07900c3d73288b0d08cfd INFO: 172.21.0.1:43284 - "POST /node/69c07900c3d73288b0d08cfd HTTP/1.0" 202 Accepted 2026-03-24 23:19:46,712 - lava_callback - INFO - Starting async job submit for node 69c07900c3d73288b0d08cfd 2026-03-24 23:19:46,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:19:46,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07900c3d73288b0d08cfd HTTP/1.1" 200 None 2026-03-24 23:19:46,729 - lava_callback - DEBUG - Processing node 69c07900c3d73288b0d08cfd: baseline-arm64-baylibre 2026-03-24 23:19:46,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07900c3d73288b0d08cfd 2026-03-24 23:19:46,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:19:46,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07900c3d73288b0d08cfd HTTP/1.1" 200 None 2026-03-24 23:19:46,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:19:46,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:19:46,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:19:46,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07900c3d73288b0d08cfd HTTP/1.1" 200 None 2026-03-24 23:19:46,792 - lava_callback - ERROR - Error processing callback for node 69c07900c3d73288b0d08cfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59630 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:20:52,652 - lava_callback - INFO - Received callback for node 69c07947c3d73288b0d08d06 INFO: 172.21.0.1:35694 - "POST /node/69c07947c3d73288b0d08d06 HTTP/1.0" 202 Accepted 2026-03-24 23:20:52,658 - lava_callback - INFO - Starting async job submit for node 69c07947c3d73288b0d08d06 2026-03-24 23:20:52,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:20:52,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07947c3d73288b0d08d06 HTTP/1.1" 200 None 2026-03-24 23:20:52,674 - lava_callback - DEBUG - Processing node 69c07947c3d73288b0d08d06: baseline-arm-clabbe 2026-03-24 23:20:52,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07947c3d73288b0d08d06 2026-03-24 23:20:52,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:20:52,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07947c3d73288b0d08d06 HTTP/1.1" 200 None 2026-03-24 23:20:52,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:20:52,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 23:20:52,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:20:52,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07947c3d73288b0d08d06 HTTP/1.1" 200 None 2026-03-24 23:20:52,755 - lava_callback - ERROR - Error processing callback for node 69c07947c3d73288b0d08d06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:21:32,627 - lava_callback - INFO - Received callback for node 69c0796fc3d73288b0d08d18 INFO: 172.21.0.1:34220 - "POST /node/69c0796fc3d73288b0d08d18 HTTP/1.0" 202 Accepted 2026-03-24 23:21:32,633 - lava_callback - INFO - Starting async job submit for node 69c0796fc3d73288b0d08d18 2026-03-24 23:21:32,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:21:32,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0796fc3d73288b0d08d18 HTTP/1.1" 200 None 2026-03-24 23:21:32,669 - lava_callback - DEBUG - Processing node 69c0796fc3d73288b0d08d18: baseline-arm-clabbe 2026-03-24 23:21:32,669 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0796fc3d73288b0d08d18 2026-03-24 23:21:32,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:21:32,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0796fc3d73288b0d08d18 HTTP/1.1" 200 None 2026-03-24 23:21:32,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:21:32,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 23:21:32,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:21:32,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0796fc3d73288b0d08d18 HTTP/1.1" 200 None 2026-03-24 23:21:32,737 - lava_callback - ERROR - Error processing callback for node 69c0796fc3d73288b0d08d18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34214 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:22:12,653 - lava_callback - INFO - Received callback for node 69c07999c3d73288b0d08d21 INFO: 172.21.0.1:55748 - "POST /node/69c07999c3d73288b0d08d21 HTTP/1.0" 202 Accepted 2026-03-24 23:22:12,659 - lava_callback - INFO - Starting async job submit for node 69c07999c3d73288b0d08d21 2026-03-24 23:22:12,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:12,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07999c3d73288b0d08d21 HTTP/1.1" 200 None 2026-03-24 23:22:12,674 - lava_callback - DEBUG - Processing node 69c07999c3d73288b0d08d21: baseline-arm-clabbe 2026-03-24 23:22:12,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07999c3d73288b0d08d21 2026-03-24 23:22:12,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:12,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07999c3d73288b0d08d21 HTTP/1.1" 200 None 2026-03-24 23:22:12,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:12,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 23:22:12,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:12,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07999c3d73288b0d08d21 HTTP/1.1" 200 None 2026-03-24 23:22:12,738 - lava_callback - ERROR - Error processing callback for node 69c07999c3d73288b0d08d21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49924 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:22:52,695 - lava_callback - INFO - Received callback for node 69c079c2c3d73288b0d08d2b INFO: 172.21.0.1:60462 - "POST /node/69c079c2c3d73288b0d08d2b HTTP/1.0" 202 Accepted 2026-03-24 23:22:52,701 - lava_callback - INFO - Starting async job submit for node 69c079c2c3d73288b0d08d2b 2026-03-24 23:22:52,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:52,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c079c2c3d73288b0d08d2b HTTP/1.1" 200 None 2026-03-24 23:22:52,718 - lava_callback - DEBUG - Processing node 69c079c2c3d73288b0d08d2b: baseline-arm-clabbe 2026-03-24 23:22:52,719 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c079c2c3d73288b0d08d2b 2026-03-24 23:22:52,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:52,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c079c2c3d73288b0d08d2b HTTP/1.1" 200 None 2026-03-24 23:22:52,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:52,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 23:22:52,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:22:52,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c079c2c3d73288b0d08d2b HTTP/1.1" 200 None 2026-03-24 23:22:52,777 - lava_callback - ERROR - Error processing callback for node 69c079c2c3d73288b0d08d2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34660 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:23:32,730 - lava_callback - INFO - Received callback for node 69c079e9c3d73288b0d08d35 INFO: 172.21.0.1:33078 - "POST /node/69c079e9c3d73288b0d08d35 HTTP/1.0" 202 Accepted 2026-03-24 23:23:32,735 - lava_callback - INFO - Starting async job submit for node 69c079e9c3d73288b0d08d35 2026-03-24 23:23:32,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:23:32,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c079e9c3d73288b0d08d35 HTTP/1.1" 200 None 2026-03-24 23:23:32,752 - lava_callback - DEBUG - Processing node 69c079e9c3d73288b0d08d35: baseline-arm-clabbe 2026-03-24 23:23:32,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c079e9c3d73288b0d08d35 2026-03-24 23:23:32,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:23:32,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c079e9c3d73288b0d08d35 HTTP/1.1" 200 None 2026-03-24 23:23:32,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:23:32,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-24 23:23:32,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:23:32,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c079e9c3d73288b0d08d35 HTTP/1.1" 200 None 2026-03-24 23:23:32,819 - lava_callback - ERROR - Error processing callback for node 69c079e9c3d73288b0d08d35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45066 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:24:52,764 - lava_callback - INFO - Received callback for node 69c07a31c3d73288b0d08d40 INFO: 172.21.0.1:40956 - "POST /node/69c07a31c3d73288b0d08d40 HTTP/1.0" 202 Accepted 2026-03-24 23:24:52,769 - lava_callback - INFO - Starting async job submit for node 69c07a31c3d73288b0d08d40 2026-03-24 23:24:52,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:24:52,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07a31c3d73288b0d08d40 HTTP/1.1" 200 None 2026-03-24 23:24:52,793 - lava_callback - DEBUG - Processing node 69c07a31c3d73288b0d08d40: baseline-arm64-clabbe 2026-03-24 23:24:52,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07a31c3d73288b0d08d40 2026-03-24 23:24:52,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:24:52,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07a31c3d73288b0d08d40 HTTP/1.1" 200 None 2026-03-24 23:24:52,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:24:52,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:24:52,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:24:52,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07a31c3d73288b0d08d40 HTTP/1.1" 200 None 2026-03-24 23:24:52,850 - lava_callback - ERROR - Error processing callback for node 69c07a31c3d73288b0d08d40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50256 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:26:12,839 - lava_callback - INFO - Received callback for node 69c07a82c3d73288b0d08d4c INFO: 172.21.0.1:58232 - "POST /node/69c07a82c3d73288b0d08d4c HTTP/1.0" 202 Accepted 2026-03-24 23:26:12,845 - lava_callback - INFO - Starting async job submit for node 69c07a82c3d73288b0d08d4c 2026-03-24 23:26:12,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:26:12,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07a82c3d73288b0d08d4c HTTP/1.1" 200 None 2026-03-24 23:26:12,863 - lava_callback - DEBUG - Processing node 69c07a82c3d73288b0d08d4c: baseline-arm64-clabbe 2026-03-24 23:26:12,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07a82c3d73288b0d08d4c 2026-03-24 23:26:12,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:26:12,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07a82c3d73288b0d08d4c HTTP/1.1" 200 None 2026-03-24 23:26:12,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:26:12,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:26:12,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:26:12,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07a82c3d73288b0d08d4c HTTP/1.1" 200 None 2026-03-24 23:26:12,945 - lava_callback - ERROR - Error processing callback for node 69c07a82c3d73288b0d08d4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52982 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:27:32,934 - lava_callback - INFO - Received callback for node 69c07ad0c3d73288b0d08d53 INFO: 172.21.0.1:58674 - "POST /node/69c07ad0c3d73288b0d08d53 HTTP/1.0" 202 Accepted 2026-03-24 23:27:32,940 - lava_callback - INFO - Starting async job submit for node 69c07ad0c3d73288b0d08d53 2026-03-24 23:27:32,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:27:32,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07ad0c3d73288b0d08d53 HTTP/1.1" 200 None 2026-03-24 23:27:32,958 - lava_callback - DEBUG - Processing node 69c07ad0c3d73288b0d08d53: baseline-arm64-clabbe 2026-03-24 23:27:32,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07ad0c3d73288b0d08d53 2026-03-24 23:27:32,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:27:32,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07ad0c3d73288b0d08d53 HTTP/1.1" 200 None 2026-03-24 23:27:32,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:27:32,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:27:32,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:27:33,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07ad0c3d73288b0d08d53 HTTP/1.1" 200 None 2026-03-24 23:27:33,016 - lava_callback - ERROR - Error processing callback for node 69c07ad0c3d73288b0d08d53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39008 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:28:52,976 - lava_callback - INFO - Received callback for node 69c07b1fc3d73288b0d08d60 INFO: 172.21.0.1:45364 - "POST /node/69c07b1fc3d73288b0d08d60 HTTP/1.0" 202 Accepted 2026-03-24 23:28:52,981 - lava_callback - INFO - Starting async job submit for node 69c07b1fc3d73288b0d08d60 2026-03-24 23:28:52,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:28:53,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07b1fc3d73288b0d08d60 HTTP/1.1" 200 None 2026-03-24 23:28:53,002 - lava_callback - DEBUG - Processing node 69c07b1fc3d73288b0d08d60: baseline-arm64-clabbe 2026-03-24 23:28:53,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07b1fc3d73288b0d08d60 2026-03-24 23:28:53,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:28:53,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07b1fc3d73288b0d08d60 HTTP/1.1" 200 None 2026-03-24 23:28:53,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:28:53,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:28:53,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:28:53,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07b1fc3d73288b0d08d60 HTTP/1.1" 200 None 2026-03-24 23:28:53,057 - lava_callback - ERROR - Error processing callback for node 69c07b1fc3d73288b0d08d60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37662 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:30:53,056 - lava_callback - INFO - Received callback for node 69c07b98c3d73288b0d08d6d INFO: 172.21.0.1:56140 - "POST /node/69c07b98c3d73288b0d08d6d HTTP/1.0" 202 Accepted 2026-03-24 23:30:53,062 - lava_callback - INFO - Starting async job submit for node 69c07b98c3d73288b0d08d6d 2026-03-24 23:30:53,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:30:53,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07b98c3d73288b0d08d6d HTTP/1.1" 200 None 2026-03-24 23:30:53,079 - lava_callback - DEBUG - Processing node 69c07b98c3d73288b0d08d6d: baseline-arm64-clabbe 2026-03-24 23:30:53,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07b98c3d73288b0d08d6d 2026-03-24 23:30:53,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:30:53,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07b98c3d73288b0d08d6d HTTP/1.1" 200 None 2026-03-24 23:30:53,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:30:53,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:30:53,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:30:53,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07b98c3d73288b0d08d6d HTTP/1.1" 200 None 2026-03-24 23:30:53,138 - lava_callback - ERROR - Error processing callback for node 69c07b98c3d73288b0d08d6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52276 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:32:13,145 - lava_callback - INFO - Received callback for node 69c07be9c3d73288b0d08d7b INFO: 172.21.0.1:55448 - "POST /node/69c07be9c3d73288b0d08d7b HTTP/1.0" 202 Accepted 2026-03-24 23:32:13,150 - lava_callback - INFO - Starting async job submit for node 69c07be9c3d73288b0d08d7b 2026-03-24 23:32:13,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:32:13,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07be9c3d73288b0d08d7b HTTP/1.1" 200 None 2026-03-24 23:32:13,167 - lava_callback - DEBUG - Processing node 69c07be9c3d73288b0d08d7b: baseline-arm64-clabbe 2026-03-24 23:32:13,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07be9c3d73288b0d08d7b 2026-03-24 23:32:13,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:32:13,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07be9c3d73288b0d08d7b HTTP/1.1" 200 None 2026-03-24 23:32:13,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:32:13,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-24 23:32:13,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:32:13,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07be9c3d73288b0d08d7b HTTP/1.1" 200 None 2026-03-24 23:32:13,232 - lava_callback - ERROR - Error processing callback for node 69c07be9c3d73288b0d08d7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37018 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:33:07,340 - lava_callback - INFO - Received callback for node 69c07c2ec3d73288b0d08d86 INFO: 172.21.0.1:37024 - "POST /node/69c07c2ec3d73288b0d08d86 HTTP/1.0" 202 Accepted 2026-03-24 23:33:07,345 - lava_callback - INFO - Starting async job submit for node 69c07c2ec3d73288b0d08d86 2026-03-24 23:33:07,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:33:07,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07c2ec3d73288b0d08d86 HTTP/1.1" 200 None 2026-03-24 23:33:07,361 - lava_callback - DEBUG - Processing node 69c07c2ec3d73288b0d08d86: baseline-arm64-baylibre 2026-03-24 23:33:07,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07c2ec3d73288b0d08d86 2026-03-24 23:33:07,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:33:07,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07c2ec3d73288b0d08d86 HTTP/1.1" 200 None 2026-03-24 23:33:07,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:33:07,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 23:33:07,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:33:07,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07c2ec3d73288b0d08d86 HTTP/1.1" 200 None 2026-03-24 23:33:07,413 - lava_callback - ERROR - Error processing callback for node 69c07c2ec3d73288b0d08d86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46520 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:34:27,435 - lava_callback - INFO - Received callback for node 69c07c7cc3d73288b0d08d96 INFO: 172.21.0.1:46524 - "POST /node/69c07c7cc3d73288b0d08d96 HTTP/1.0" 202 Accepted 2026-03-24 23:34:27,441 - lava_callback - INFO - Starting async job submit for node 69c07c7cc3d73288b0d08d96 2026-03-24 23:34:27,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:34:27,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07c7cc3d73288b0d08d96 HTTP/1.1" 200 None 2026-03-24 23:34:27,457 - lava_callback - DEBUG - Processing node 69c07c7cc3d73288b0d08d96: baseline-arm64-baylibre 2026-03-24 23:34:27,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07c7cc3d73288b0d08d96 2026-03-24 23:34:27,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:34:27,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07c7cc3d73288b0d08d96 HTTP/1.1" 200 None 2026-03-24 23:34:27,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:34:27,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 23:34:27,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:34:27,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07c7cc3d73288b0d08d96 HTTP/1.1" 200 None 2026-03-24 23:34:27,536 - lava_callback - ERROR - Error processing callback for node 69c07c7cc3d73288b0d08d96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49258 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:35:47,457 - lava_callback - INFO - Received callback for node 69c07cd1c3d73288b0d08da1 INFO: 172.21.0.1:49274 - "POST /node/69c07cd1c3d73288b0d08da1 HTTP/1.0" 202 Accepted 2026-03-24 23:35:47,463 - lava_callback - INFO - Starting async job submit for node 69c07cd1c3d73288b0d08da1 2026-03-24 23:35:47,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:35:47,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07cd1c3d73288b0d08da1 HTTP/1.1" 200 None 2026-03-24 23:35:47,483 - lava_callback - DEBUG - Processing node 69c07cd1c3d73288b0d08da1: baseline-arm64-baylibre 2026-03-24 23:35:47,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07cd1c3d73288b0d08da1 2026-03-24 23:35:47,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:35:47,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07cd1c3d73288b0d08da1 HTTP/1.1" 200 None 2026-03-24 23:35:47,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:35:47,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 23:35:47,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:35:47,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07cd1c3d73288b0d08da1 HTTP/1.1" 200 None 2026-03-24 23:35:47,570 - lava_callback - ERROR - Error processing callback for node 69c07cd1c3d73288b0d08da1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38386 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:43:28,027 - lava_callback - INFO - Received callback for node 69c07e8fc3d73288b0d08e19 INFO: 172.21.0.1:38388 - "POST /node/69c07e8fc3d73288b0d08e19 HTTP/1.0" 202 Accepted 2026-03-24 23:43:28,033 - lava_callback - INFO - Starting async job submit for node 69c07e8fc3d73288b0d08e19 2026-03-24 23:43:28,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:43:28,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07e8fc3d73288b0d08e19 HTTP/1.1" 200 None 2026-03-24 23:43:28,051 - lava_callback - DEBUG - Processing node 69c07e8fc3d73288b0d08e19: baseline-arm64-baylibre 2026-03-24 23:43:28,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07e8fc3d73288b0d08e19 2026-03-24 23:43:28,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:43:28,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07e8fc3d73288b0d08e19 HTTP/1.1" 200 None 2026-03-24 23:43:28,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:43:28,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 23:43:28,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:43:28,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07e8fc3d73288b0d08e19 HTTP/1.1" 200 None 2026-03-24 23:43:28,116 - lava_callback - ERROR - Error processing callback for node 69c07e8fc3d73288b0d08e19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43954 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:44:41,966 - lava_callback - INFO - Received callback for node 69c2c2da6778148209fd5e70 INFO: 172.21.0.1:60988 - "POST /node/69c2c2da6778148209fd5e70 HTTP/1.0" 202 Accepted 2026-03-24 23:44:41,972 - lava_callback - INFO - Starting async job submit for node 69c2c2da6778148209fd5e70 2026-03-24 23:44:41,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:42,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2da6778148209fd5e70 HTTP/1.1" 200 None 2026-03-24 23:44:42,009 - lava_callback - DEBUG - Processing node 69c2c2da6778148209fd5e70: baseline-arm64-broonie 2026-03-24 23:44:42,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 23:44:42,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 23:44:42,403 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2da6778148209fd5e70/log.txt.gz 2026-03-24 23:44:42,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 23:44:42,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 23:44:42,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2da6778148209fd5e70/lava-logs.yaml 2026-03-24 23:44:42,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 23:44:42,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 23:44:42,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2c2da6778148209fd5e70/lava_callback.json.gz 2026-03-24 23:44:42,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:42,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2c2da6778148209fd5e70 HTTP/1.1" 200 None 2026-03-24 23:44:42,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:42,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 23:44:42,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:43,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2c2da6778148209fd5e70 HTTP/1.1" 200 None 2026-03-24 23:44:43,017 - lava_callback - INFO - Completed processing callback for node 69c2c2da6778148209fd5e70 INFO: 172.21.0.1:32772 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:44:47,954 - lava_callback - INFO - Received callback for node 69c07edcc3d73288b0d08e24 INFO: 172.21.0.1:32780 - "POST /node/69c07edcc3d73288b0d08e24 HTTP/1.0" 202 Accepted 2026-03-24 23:44:47,960 - lava_callback - INFO - Starting async job submit for node 69c07edcc3d73288b0d08e24 2026-03-24 23:44:47,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:47,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07edcc3d73288b0d08e24 HTTP/1.1" 200 None 2026-03-24 23:44:47,976 - lava_callback - DEBUG - Processing node 69c07edcc3d73288b0d08e24: baseline-arm64-baylibre 2026-03-24 23:44:47,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07edcc3d73288b0d08e24 2026-03-24 23:44:47,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:47,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07edcc3d73288b0d08e24 HTTP/1.1" 200 None 2026-03-24 23:44:47,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:48,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:44:48,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:44:48,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07edcc3d73288b0d08e24 HTTP/1.1" 200 None 2026-03-24 23:44:48,032 - lava_callback - ERROR - Error processing callback for node 69c07edcc3d73288b0d08e24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47666 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:45:05,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:45:05,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44948 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:46:02,071 - lava_callback - INFO - Received callback for node 69c31c026778148209fd6e06 INFO: 172.21.0.1:44492 - "POST /node/69c31c026778148209fd6e06 HTTP/1.0" 202 Accepted 2026-03-24 23:46:02,077 - lava_callback - INFO - Starting async job submit for node 69c31c026778148209fd6e06 2026-03-24 23:46:02,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:02,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31c026778148209fd6e06 HTTP/1.1" 200 None 2026-03-24 23:46:02,114 - lava_callback - DEBUG - Processing node 69c31c026778148209fd6e06: baseline-arm64-broonie 2026-03-24 23:46:02,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 23:46:02,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 23:46:02,486 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c31c026778148209fd6e06/log.txt.gz 2026-03-24 23:46:02,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 23:46:02,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 23:46:02,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c31c026778148209fd6e06/lava-logs.yaml 2026-03-24 23:46:02,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 23:46:03,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 23:46:03,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c31c026778148209fd6e06/lava_callback.json.gz 2026-03-24 23:46:03,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:03,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c31c026778148209fd6e06 HTTP/1.1" 200 None 2026-03-24 23:46:03,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:03,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-24 23:46:03,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:03,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c31c026778148209fd6e06 HTTP/1.1" 200 None 2026-03-24 23:46:03,094 - lava_callback - INFO - Completed processing callback for node 69c31c026778148209fd6e06 2026-03-24 23:46:05,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:05,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44502 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:46:08,004 - lava_callback - INFO - Received callback for node 69c07f2bc3d73288b0d08e32 INFO: 172.21.0.1:44510 - "POST /node/69c07f2bc3d73288b0d08e32 HTTP/1.0" 202 Accepted 2026-03-24 23:46:08,011 - lava_callback - INFO - Starting async job submit for node 69c07f2bc3d73288b0d08e32 2026-03-24 23:46:08,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:08,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07f2bc3d73288b0d08e32 HTTP/1.1" 200 None 2026-03-24 23:46:08,031 - lava_callback - DEBUG - Processing node 69c07f2bc3d73288b0d08e32: baseline-arm64-baylibre 2026-03-24 23:46:08,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07f2bc3d73288b0d08e32 2026-03-24 23:46:08,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:08,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07f2bc3d73288b0d08e32 HTTP/1.1" 200 None 2026-03-24 23:46:08,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:08,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:46:08,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:46:08,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07f2bc3d73288b0d08e32 HTTP/1.1" 200 None 2026-03-24 23:46:08,088 - lava_callback - ERROR - Error processing callback for node 69c07f2bc3d73288b0d08e32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:47:08,045 - lava_callback - INFO - Received callback for node 69c07f79c3d73288b0d08e3c INFO: 172.21.0.1:55078 - "POST /node/69c07f79c3d73288b0d08e3c HTTP/1.0" 202 Accepted 2026-03-24 23:47:08,050 - lava_callback - INFO - Starting async job submit for node 69c07f79c3d73288b0d08e3c 2026-03-24 23:47:08,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:47:08,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07f79c3d73288b0d08e3c HTTP/1.1" 200 None 2026-03-24 23:47:08,066 - lava_callback - DEBUG - Processing node 69c07f79c3d73288b0d08e3c: baseline-arm64-baylibre 2026-03-24 23:47:08,067 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07f79c3d73288b0d08e3c 2026-03-24 23:47:08,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:47:08,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07f79c3d73288b0d08e3c HTTP/1.1" 200 None 2026-03-24 23:47:08,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:47:08,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:47:08,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:47:08,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07f79c3d73288b0d08e3c HTTP/1.1" 200 None 2026-03-24 23:47:08,126 - lava_callback - ERROR - Error processing callback for node 69c07f79c3d73288b0d08e3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59850 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:48:28,170 - lava_callback - INFO - Received callback for node 69c07fc6c3d73288b0d08e4b INFO: 172.21.0.1:59854 - "POST /node/69c07fc6c3d73288b0d08e4b HTTP/1.0" 202 Accepted 2026-03-24 23:48:28,176 - lava_callback - INFO - Starting async job submit for node 69c07fc6c3d73288b0d08e4b 2026-03-24 23:48:28,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:48:28,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07fc6c3d73288b0d08e4b HTTP/1.1" 200 None 2026-03-24 23:48:28,199 - lava_callback - DEBUG - Processing node 69c07fc6c3d73288b0d08e4b: baseline-arm64-baylibre 2026-03-24 23:48:28,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c07fc6c3d73288b0d08e4b 2026-03-24 23:48:28,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:48:28,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c07fc6c3d73288b0d08e4b HTTP/1.1" 200 None 2026-03-24 23:48:28,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:48:28,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:48:28,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:48:28,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c07fc6c3d73288b0d08e4b HTTP/1.1" 200 None 2026-03-24 23:48:28,274 - lava_callback - ERROR - Error processing callback for node 69c07fc6c3d73288b0d08e4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53620 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:49:48,237 - lava_callback - INFO - Received callback for node 69c08012c3d73288b0d08e52 INFO: 172.21.0.1:53628 - "POST /node/69c08012c3d73288b0d08e52 HTTP/1.0" 202 Accepted 2026-03-24 23:49:48,244 - lava_callback - INFO - Starting async job submit for node 69c08012c3d73288b0d08e52 2026-03-24 23:49:48,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:49:48,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08012c3d73288b0d08e52 HTTP/1.1" 200 None 2026-03-24 23:49:48,262 - lava_callback - DEBUG - Processing node 69c08012c3d73288b0d08e52: baseline-arm64-baylibre 2026-03-24 23:49:48,262 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08012c3d73288b0d08e52 2026-03-24 23:49:48,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:49:48,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08012c3d73288b0d08e52 HTTP/1.1" 200 None 2026-03-24 23:49:48,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:49:48,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:49:48,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:49:48,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08012c3d73288b0d08e52 HTTP/1.1" 200 None 2026-03-24 23:49:48,329 - lava_callback - ERROR - Error processing callback for node 69c08012c3d73288b0d08e52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49994 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:51:08,447 - lava_callback - INFO - Received callback for node 69c0805bc3d73288b0d08e5e INFO: 172.21.0.1:49998 - "POST /node/69c0805bc3d73288b0d08e5e HTTP/1.0" 202 Accepted 2026-03-24 23:51:08,453 - lava_callback - INFO - Starting async job submit for node 69c0805bc3d73288b0d08e5e 2026-03-24 23:51:08,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:51:08,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0805bc3d73288b0d08e5e HTTP/1.1" 200 None 2026-03-24 23:51:08,475 - lava_callback - DEBUG - Processing node 69c0805bc3d73288b0d08e5e: baseline-arm64-baylibre 2026-03-24 23:51:08,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0805bc3d73288b0d08e5e 2026-03-24 23:51:08,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:51:08,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0805bc3d73288b0d08e5e HTTP/1.1" 200 None 2026-03-24 23:51:08,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:51:08,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:51:08,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:51:08,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0805bc3d73288b0d08e5e HTTP/1.1" 200 None 2026-03-24 23:51:08,545 - lava_callback - ERROR - Error processing callback for node 69c0805bc3d73288b0d08e5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33940 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:52:14,055 - lava_callback - INFO - Received callback for node 69c080a7c3d73288b0d08e67 INFO: 172.21.0.1:44562 - "POST /node/69c080a7c3d73288b0d08e67 HTTP/1.0" 202 Accepted 2026-03-24 23:52:14,061 - lava_callback - INFO - Starting async job submit for node 69c080a7c3d73288b0d08e67 2026-03-24 23:52:14,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:52:14,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c080a7c3d73288b0d08e67 HTTP/1.1" 200 None 2026-03-24 23:52:14,078 - lava_callback - DEBUG - Processing node 69c080a7c3d73288b0d08e67: baseline-arm64-clabbe 2026-03-24 23:52:14,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c080a7c3d73288b0d08e67 2026-03-24 23:52:14,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:52:14,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c080a7c3d73288b0d08e67 HTTP/1.1" 200 None 2026-03-24 23:52:14,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:52:14,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:52:14,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:52:14,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c080a7c3d73288b0d08e67 HTTP/1.1" 200 None 2026-03-24 23:52:14,150 - lava_callback - ERROR - Error processing callback for node 69c080a7c3d73288b0d08e67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43776 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:53:34,060 - lava_callback - INFO - Received callback for node 69c080f1c3d73288b0d08e73 INFO: 172.21.0.1:35434 - "POST /node/69c080f1c3d73288b0d08e73 HTTP/1.0" 202 Accepted 2026-03-24 23:53:34,066 - lava_callback - INFO - Starting async job submit for node 69c080f1c3d73288b0d08e73 2026-03-24 23:53:34,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:53:34,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c080f1c3d73288b0d08e73 HTTP/1.1" 200 None 2026-03-24 23:53:34,087 - lava_callback - DEBUG - Processing node 69c080f1c3d73288b0d08e73: baseline-arm64-clabbe 2026-03-24 23:53:34,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c080f1c3d73288b0d08e73 2026-03-24 23:53:34,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:53:34,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c080f1c3d73288b0d08e73 HTTP/1.1" 200 None 2026-03-24 23:53:34,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:53:34,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:53:34,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:53:34,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c080f1c3d73288b0d08e73 HTTP/1.1" 200 None 2026-03-24 23:53:34,146 - lava_callback - ERROR - Error processing callback for node 69c080f1c3d73288b0d08e73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:54:54,132 - lava_callback - INFO - Received callback for node 69c0813cc3d73288b0d08e7c INFO: 172.21.0.1:47110 - "POST /node/69c0813cc3d73288b0d08e7c HTTP/1.0" 202 Accepted 2026-03-24 23:54:54,137 - lava_callback - INFO - Starting async job submit for node 69c0813cc3d73288b0d08e7c 2026-03-24 23:54:54,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:54:54,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0813cc3d73288b0d08e7c HTTP/1.1" 200 None 2026-03-24 23:54:54,153 - lava_callback - DEBUG - Processing node 69c0813cc3d73288b0d08e7c: baseline-arm64-clabbe 2026-03-24 23:54:54,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0813cc3d73288b0d08e7c 2026-03-24 23:54:54,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:54:54,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0813cc3d73288b0d08e7c HTTP/1.1" 200 None 2026-03-24 23:54:54,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:54:54,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:54:54,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:54:54,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0813cc3d73288b0d08e7c HTTP/1.1" 200 None 2026-03-24 23:54:54,217 - lava_callback - ERROR - Error processing callback for node 69c0813cc3d73288b0d08e7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:56:14,242 - lava_callback - INFO - Received callback for node 69c08189c3d73288b0d08e89 INFO: 172.21.0.1:59648 - "POST /node/69c08189c3d73288b0d08e89 HTTP/1.0" 202 Accepted 2026-03-24 23:56:14,248 - lava_callback - INFO - Starting async job submit for node 69c08189c3d73288b0d08e89 2026-03-24 23:56:14,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:56:14,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08189c3d73288b0d08e89 HTTP/1.1" 200 None 2026-03-24 23:56:14,264 - lava_callback - DEBUG - Processing node 69c08189c3d73288b0d08e89: baseline-arm64-clabbe 2026-03-24 23:56:14,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08189c3d73288b0d08e89 2026-03-24 23:56:14,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:56:14,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08189c3d73288b0d08e89 HTTP/1.1" 200 None 2026-03-24 23:56:14,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:56:14,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:56:14,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:56:14,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08189c3d73288b0d08e89 HTTP/1.1" 200 None 2026-03-24 23:56:14,329 - lava_callback - ERROR - Error processing callback for node 69c08189c3d73288b0d08e89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40436 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:57:14,325 - lava_callback - INFO - Received callback for node 69c081d6c3d73288b0d08e8e INFO: 172.21.0.1:49446 - "POST /node/69c081d6c3d73288b0d08e8e HTTP/1.0" 202 Accepted 2026-03-24 23:57:14,330 - lava_callback - INFO - Starting async job submit for node 69c081d6c3d73288b0d08e8e 2026-03-24 23:57:14,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:57:14,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c081d6c3d73288b0d08e8e HTTP/1.1" 200 None 2026-03-24 23:57:14,351 - lava_callback - DEBUG - Processing node 69c081d6c3d73288b0d08e8e: baseline-arm64-clabbe 2026-03-24 23:57:14,351 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c081d6c3d73288b0d08e8e 2026-03-24 23:57:14,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:57:14,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c081d6c3d73288b0d08e8e HTTP/1.1" 200 None 2026-03-24 23:57:14,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:57:14,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:57:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:57:14,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c081d6c3d73288b0d08e8e HTTP/1.1" 200 None 2026-03-24 23:57:14,409 - lava_callback - ERROR - Error processing callback for node 69c081d6c3d73288b0d08e8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41278 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:58:34,337 - lava_callback - INFO - Received callback for node 69c08222c3d73288b0d08e9b INFO: 172.21.0.1:41214 - "POST /node/69c08222c3d73288b0d08e9b HTTP/1.0" 202 Accepted 2026-03-24 23:58:34,342 - lava_callback - INFO - Starting async job submit for node 69c08222c3d73288b0d08e9b 2026-03-24 23:58:34,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:58:34,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08222c3d73288b0d08e9b HTTP/1.1" 200 None 2026-03-24 23:58:34,359 - lava_callback - DEBUG - Processing node 69c08222c3d73288b0d08e9b: baseline-arm64-clabbe 2026-03-24 23:58:34,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08222c3d73288b0d08e9b 2026-03-24 23:58:34,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:58:34,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08222c3d73288b0d08e9b HTTP/1.1" 200 None 2026-03-24 23:58:34,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:58:34,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-24 23:58:34,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:58:34,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08222c3d73288b0d08e9b HTTP/1.1" 200 None 2026-03-24 23:58:34,416 - lava_callback - ERROR - Error processing callback for node 69c08222c3d73288b0d08e9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40866 - "GET / HTTP/1.0" 200 OK 2026-03-24 23:59:52,783 - lava_callback - INFO - Received callback for node 69c08269c3d73288b0d08ea2 INFO: 172.21.0.1:42158 - "POST /node/69c08269c3d73288b0d08ea2 HTTP/1.0" 202 Accepted 2026-03-24 23:59:52,788 - lava_callback - INFO - Starting async job submit for node 69c08269c3d73288b0d08ea2 2026-03-24 23:59:52,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:59:52,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08269c3d73288b0d08ea2 HTTP/1.1" 200 None 2026-03-24 23:59:52,834 - lava_callback - DEBUG - Processing node 69c08269c3d73288b0d08ea2: baseline-arm64-baylibre 2026-03-24 23:59:52,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08269c3d73288b0d08ea2 2026-03-24 23:59:52,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:59:52,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08269c3d73288b0d08ea2 HTTP/1.1" 200 None 2026-03-24 23:59:52,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:59:52,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-24 23:59:52,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 23:59:52,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08269c3d73288b0d08ea2 HTTP/1.1" 200 None 2026-03-24 23:59:52,923 - lava_callback - ERROR - Error processing callback for node 69c08269c3d73288b0d08ea2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48156 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:01:12,799 - lava_callback - INFO - Received callback for node 69c082b3c3d73288b0d08eaf INFO: 172.21.0.1:40202 - "POST /node/69c082b3c3d73288b0d08eaf HTTP/1.0" 202 Accepted 2026-03-25 00:01:12,806 - lava_callback - INFO - Starting async job submit for node 69c082b3c3d73288b0d08eaf 2026-03-25 00:01:12,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:01:12,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c082b3c3d73288b0d08eaf HTTP/1.1" 200 None 2026-03-25 00:01:12,827 - lava_callback - DEBUG - Processing node 69c082b3c3d73288b0d08eaf: baseline-arm64-baylibre 2026-03-25 00:01:12,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c082b3c3d73288b0d08eaf 2026-03-25 00:01:12,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:01:12,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c082b3c3d73288b0d08eaf HTTP/1.1" 200 None 2026-03-25 00:01:12,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:01:12,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:01:12,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:01:12,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c082b3c3d73288b0d08eaf HTTP/1.1" 200 None 2026-03-25 00:01:12,913 - lava_callback - ERROR - Error processing callback for node 69c082b3c3d73288b0d08eaf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57492 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:02:34,514 - lava_callback - INFO - Received callback for node 69c08309c3d73288b0d08eb9 INFO: 172.21.0.1:57498 - "POST /node/69c08309c3d73288b0d08eb9 HTTP/1.0" 202 Accepted 2026-03-25 00:02:34,519 - lava_callback - INFO - Starting async job submit for node 69c08309c3d73288b0d08eb9 2026-03-25 00:02:34,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:02:34,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08309c3d73288b0d08eb9 HTTP/1.1" 200 None 2026-03-25 00:02:34,548 - lava_callback - DEBUG - Processing node 69c08309c3d73288b0d08eb9: baseline-arm64-clabbe 2026-03-25 00:02:34,548 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08309c3d73288b0d08eb9 2026-03-25 00:02:34,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:02:34,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08309c3d73288b0d08eb9 HTTP/1.1" 200 None 2026-03-25 00:02:34,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:02:34,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:02:34,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:02:34,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08309c3d73288b0d08eb9 HTTP/1.1" 200 None 2026-03-25 00:02:34,600 - lava_callback - ERROR - Error processing callback for node 69c08309c3d73288b0d08eb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56632 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:03:54,541 - lava_callback - INFO - Received callback for node 69c08356c3d73288b0d08ec0 INFO: 172.21.0.1:43054 - "POST /node/69c08356c3d73288b0d08ec0 HTTP/1.0" 202 Accepted 2026-03-25 00:03:54,547 - lava_callback - INFO - Starting async job submit for node 69c08356c3d73288b0d08ec0 2026-03-25 00:03:54,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:03:54,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08356c3d73288b0d08ec0 HTTP/1.1" 200 None 2026-03-25 00:03:54,563 - lava_callback - DEBUG - Processing node 69c08356c3d73288b0d08ec0: baseline-arm64-clabbe 2026-03-25 00:03:54,564 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08356c3d73288b0d08ec0 2026-03-25 00:03:54,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:03:54,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08356c3d73288b0d08ec0 HTTP/1.1" 200 None 2026-03-25 00:03:54,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:03:54,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:03:54,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:03:54,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08356c3d73288b0d08ec0 HTTP/1.1" 200 None 2026-03-25 00:03:54,624 - lava_callback - ERROR - Error processing callback for node 69c08356c3d73288b0d08ec0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42138 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:04:54,612 - lava_callback - INFO - Received callback for node 69c08395c3d73288b0d08ec8 INFO: 172.21.0.1:33190 - "POST /node/69c08395c3d73288b0d08ec8 HTTP/1.0" 202 Accepted 2026-03-25 00:04:54,618 - lava_callback - INFO - Starting async job submit for node 69c08395c3d73288b0d08ec8 2026-03-25 00:04:54,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:04:54,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08395c3d73288b0d08ec8 HTTP/1.1" 200 None 2026-03-25 00:04:54,638 - lava_callback - DEBUG - Processing node 69c08395c3d73288b0d08ec8: baseline-arm-clabbe 2026-03-25 00:04:54,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08395c3d73288b0d08ec8 2026-03-25 00:04:54,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:04:54,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08395c3d73288b0d08ec8 HTTP/1.1" 200 None 2026-03-25 00:04:54,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:04:54,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 00:04:54,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:04:54,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08395c3d73288b0d08ec8 HTTP/1.1" 200 None 2026-03-25 00:04:54,725 - lava_callback - ERROR - Error processing callback for node 69c08395c3d73288b0d08ec8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49234 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:05:34,587 - lava_callback - INFO - Received callback for node 69c083b9c3d73288b0d08ed4 INFO: 172.21.0.1:49248 - "POST /node/69c083b9c3d73288b0d08ed4 HTTP/1.0" 202 Accepted 2026-03-25 00:05:34,592 - lava_callback - INFO - Starting async job submit for node 69c083b9c3d73288b0d08ed4 2026-03-25 00:05:34,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:34,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c083b9c3d73288b0d08ed4 HTTP/1.1" 200 None 2026-03-25 00:05:34,608 - lava_callback - DEBUG - Processing node 69c083b9c3d73288b0d08ed4: baseline-arm-clabbe 2026-03-25 00:05:34,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c083b9c3d73288b0d08ed4 2026-03-25 00:05:34,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:34,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c083b9c3d73288b0d08ed4 HTTP/1.1" 200 None 2026-03-25 00:05:34,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:34,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 00:05:34,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:34,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c083b9c3d73288b0d08ed4 HTTP/1.1" 200 None 2026-03-25 00:05:34,660 - lava_callback - ERROR - Error processing callback for node 69c083b9c3d73288b0d08ed4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:05:54,630 - lava_callback - INFO - Received callback for node 69c083ddc3d73288b0d08edf INFO: 172.21.0.1:52926 - "POST /node/69c083ddc3d73288b0d08edf HTTP/1.0" 202 Accepted 2026-03-25 00:05:54,636 - lava_callback - INFO - Starting async job submit for node 69c083ddc3d73288b0d08edf 2026-03-25 00:05:54,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:54,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c083ddc3d73288b0d08edf HTTP/1.1" 200 None 2026-03-25 00:05:54,651 - lava_callback - DEBUG - Processing node 69c083ddc3d73288b0d08edf: baseline-arm-clabbe 2026-03-25 00:05:54,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c083ddc3d73288b0d08edf 2026-03-25 00:05:54,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:54,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c083ddc3d73288b0d08edf HTTP/1.1" 200 None 2026-03-25 00:05:54,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:54,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 00:05:54,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:05:54,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c083ddc3d73288b0d08edf HTTP/1.1" 200 None 2026-03-25 00:05:54,714 - lava_callback - ERROR - Error processing callback for node 69c083ddc3d73288b0d08edf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:06:34,727 - lava_callback - INFO - Received callback for node 69c08401c3d73288b0d08ee8 INFO: 172.21.0.1:52690 - "POST /node/69c08401c3d73288b0d08ee8 HTTP/1.0" 202 Accepted 2026-03-25 00:06:34,733 - lava_callback - INFO - Starting async job submit for node 69c08401c3d73288b0d08ee8 2026-03-25 00:06:34,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:06:34,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08401c3d73288b0d08ee8 HTTP/1.1" 200 None 2026-03-25 00:06:34,749 - lava_callback - DEBUG - Processing node 69c08401c3d73288b0d08ee8: baseline-arm-clabbe 2026-03-25 00:06:34,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08401c3d73288b0d08ee8 2026-03-25 00:06:34,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:06:34,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08401c3d73288b0d08ee8 HTTP/1.1" 200 None 2026-03-25 00:06:34,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:06:34,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 00:06:34,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:06:34,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08401c3d73288b0d08ee8 HTTP/1.1" 200 None 2026-03-25 00:06:34,813 - lava_callback - ERROR - Error processing callback for node 69c08401c3d73288b0d08ee8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56030 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:07:14,665 - lava_callback - INFO - Received callback for node 69c08425c3d73288b0d08ef1 INFO: 172.21.0.1:49402 - "POST /node/69c08425c3d73288b0d08ef1 HTTP/1.0" 202 Accepted 2026-03-25 00:07:14,671 - lava_callback - INFO - Starting async job submit for node 69c08425c3d73288b0d08ef1 2026-03-25 00:07:14,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:14,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08425c3d73288b0d08ef1 HTTP/1.1" 200 None 2026-03-25 00:07:14,687 - lava_callback - DEBUG - Processing node 69c08425c3d73288b0d08ef1: baseline-arm-clabbe 2026-03-25 00:07:14,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08425c3d73288b0d08ef1 2026-03-25 00:07:14,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:14,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08425c3d73288b0d08ef1 HTTP/1.1" 200 None 2026-03-25 00:07:14,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:14,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 00:07:14,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:14,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08425c3d73288b0d08ef1 HTTP/1.1" 200 None 2026-03-25 00:07:14,739 - lava_callback - ERROR - Error processing callback for node 69c08425c3d73288b0d08ef1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60510 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:07:54,666 - lava_callback - INFO - Received callback for node 69c08457c3d73288b0d08eff INFO: 172.21.0.1:54946 - "POST /node/69c08457c3d73288b0d08eff HTTP/1.0" 202 Accepted 2026-03-25 00:07:54,671 - lava_callback - INFO - Starting async job submit for node 69c08457c3d73288b0d08eff 2026-03-25 00:07:54,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:54,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08457c3d73288b0d08eff HTTP/1.1" 200 None 2026-03-25 00:07:54,688 - lava_callback - DEBUG - Processing node 69c08457c3d73288b0d08eff: baseline-arm64-clabbe 2026-03-25 00:07:54,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08457c3d73288b0d08eff 2026-03-25 00:07:54,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:54,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08457c3d73288b0d08eff HTTP/1.1" 200 None 2026-03-25 00:07:54,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:54,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:07:54,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:07:54,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08457c3d73288b0d08eff HTTP/1.1" 200 None 2026-03-25 00:07:54,749 - lava_callback - ERROR - Error processing callback for node 69c08457c3d73288b0d08eff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:09:14,735 - lava_callback - INFO - Received callback for node 69c084a3c3d73288b0d08f06 INFO: 172.21.0.1:48144 - "POST /node/69c084a3c3d73288b0d08f06 HTTP/1.0" 202 Accepted 2026-03-25 00:09:14,740 - lava_callback - INFO - Starting async job submit for node 69c084a3c3d73288b0d08f06 2026-03-25 00:09:14,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:09:14,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c084a3c3d73288b0d08f06 HTTP/1.1" 200 None 2026-03-25 00:09:14,765 - lava_callback - DEBUG - Processing node 69c084a3c3d73288b0d08f06: baseline-arm64-clabbe 2026-03-25 00:09:14,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c084a3c3d73288b0d08f06 2026-03-25 00:09:14,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:09:14,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c084a3c3d73288b0d08f06 HTTP/1.1" 200 None 2026-03-25 00:09:14,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:09:14,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:09:14,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:09:14,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c084a3c3d73288b0d08f06 HTTP/1.1" 200 None 2026-03-25 00:09:14,828 - lava_callback - ERROR - Error processing callback for node 69c084a3c3d73288b0d08f06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45096 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:10:34,781 - lava_callback - INFO - Received callback for node 69c084ebc3d73288b0d08f13 INFO: 172.21.0.1:45108 - "POST /node/69c084ebc3d73288b0d08f13 HTTP/1.0" 202 Accepted 2026-03-25 00:10:34,786 - lava_callback - INFO - Starting async job submit for node 69c084ebc3d73288b0d08f13 2026-03-25 00:10:34,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:10:34,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c084ebc3d73288b0d08f13 HTTP/1.1" 200 None 2026-03-25 00:10:34,803 - lava_callback - DEBUG - Processing node 69c084ebc3d73288b0d08f13: baseline-arm64-clabbe 2026-03-25 00:10:34,803 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c084ebc3d73288b0d08f13 2026-03-25 00:10:34,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:10:34,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c084ebc3d73288b0d08f13 HTTP/1.1" 200 None 2026-03-25 00:10:34,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:10:34,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:10:34,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:10:34,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c084ebc3d73288b0d08f13 HTTP/1.1" 200 None 2026-03-25 00:10:34,871 - lava_callback - ERROR - Error processing callback for node 69c084ebc3d73288b0d08f13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59816 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:11:34,826 - lava_callback - INFO - Received callback for node 69c08533c3d73288b0d08f1a INFO: 172.21.0.1:59832 - "POST /node/69c08533c3d73288b0d08f1a HTTP/1.0" 202 Accepted 2026-03-25 00:11:34,835 - lava_callback - INFO - Starting async job submit for node 69c08533c3d73288b0d08f1a 2026-03-25 00:11:34,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:11:34,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08533c3d73288b0d08f1a HTTP/1.1" 200 None 2026-03-25 00:11:34,871 - lava_callback - DEBUG - Processing node 69c08533c3d73288b0d08f1a: baseline-arm64-clabbe 2026-03-25 00:11:34,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08533c3d73288b0d08f1a 2026-03-25 00:11:34,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:11:34,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08533c3d73288b0d08f1a HTTP/1.1" 200 None 2026-03-25 00:11:34,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:11:34,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:11:34,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:11:34,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08533c3d73288b0d08f1a HTTP/1.1" 200 None 2026-03-25 00:11:34,950 - lava_callback - ERROR - Error processing callback for node 69c08533c3d73288b0d08f1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]