INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45944 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:26:43,249 - lava_callback - INFO - Received callback for node 69f6963e1f903fec0b0dbce8 2026-05-05 00:26:43,255 - lava_callback - INFO - Starting async job submit for node 69f6963e1f903fec0b0dbce8 INFO: 172.20.0.1:46908 - "POST /node/69f6963e1f903fec0b0dbce8 HTTP/1.0" 202 Accepted 2026-05-05 00:26:43,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:26:43,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6963e1f903fec0b0dbce8 HTTP/1.1" 200 None 2026-05-05 00:26:43,276 - lava_callback - DEBUG - Processing node 69f6963e1f903fec0b0dbce8: baseline-arm-clabbe 2026-05-05 00:26:43,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6963e1f903fec0b0dbce8 2026-05-05 00:26:43,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:26:43,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6963e1f903fec0b0dbce8 HTTP/1.1" 200 None 2026-05-05 00:26:43,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:26:43,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:26:43,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:26:43,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6963e1f903fec0b0dbce8 HTTP/1.1" 200 None 2026-05-05 00:26:43,323 - lava_callback - ERROR - Error processing callback for node 69f6963e1f903fec0b0dbce8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59142 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:28:43,080 - lava_callback - INFO - Received callback for node 69f696b11f903fec0b0dbd27 INFO: 172.20.0.1:38100 - "POST /node/69f696b11f903fec0b0dbd27 HTTP/1.0" 202 Accepted 2026-05-05 00:28:43,086 - lava_callback - INFO - Starting async job submit for node 69f696b11f903fec0b0dbd27 2026-05-05 00:28:43,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:28:43,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f696b11f903fec0b0dbd27 HTTP/1.1" 200 None 2026-05-05 00:28:43,101 - lava_callback - DEBUG - Processing node 69f696b11f903fec0b0dbd27: baseline-arm-clabbe 2026-05-05 00:28:43,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f696b11f903fec0b0dbd27 2026-05-05 00:28:43,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:28:43,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f696b11f903fec0b0dbd27 HTTP/1.1" 200 None 2026-05-05 00:28:43,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:28:43,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:28:43,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:28:43,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f696b11f903fec0b0dbd27 HTTP/1.1" 200 None 2026-05-05 00:28:43,147 - lava_callback - ERROR - Error processing callback for node 69f696b11f903fec0b0dbd27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48088 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:29:23,161 - lava_callback - INFO - Received callback for node 69f696d71f903fec0b0dbd3a INFO: 172.20.0.1:52836 - "POST /node/69f696d71f903fec0b0dbd3a HTTP/1.0" 202 Accepted 2026-05-05 00:29:23,167 - lava_callback - INFO - Starting async job submit for node 69f696d71f903fec0b0dbd3a 2026-05-05 00:29:23,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:29:23,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f696d71f903fec0b0dbd3a HTTP/1.1" 200 None 2026-05-05 00:29:23,181 - lava_callback - DEBUG - Processing node 69f696d71f903fec0b0dbd3a: baseline-arm-clabbe 2026-05-05 00:29:23,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f696d71f903fec0b0dbd3a 2026-05-05 00:29:23,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:29:23,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f696d71f903fec0b0dbd3a HTTP/1.1" 200 None 2026-05-05 00:29:23,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:29:23,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:29:23,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:29:23,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f696d71f903fec0b0dbd3a HTTP/1.1" 200 None 2026-05-05 00:29:23,241 - lava_callback - ERROR - Error processing callback for node 69f696d71f903fec0b0dbd3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57244 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:30:03,110 - lava_callback - INFO - Received callback for node 69f696fb1f903fec0b0dbd52 INFO: 172.20.0.1:38422 - "POST /node/69f696fb1f903fec0b0dbd52 HTTP/1.0" 202 Accepted 2026-05-05 00:30:03,115 - lava_callback - INFO - Starting async job submit for node 69f696fb1f903fec0b0dbd52 2026-05-05 00:30:03,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:30:03,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f696fb1f903fec0b0dbd52 HTTP/1.1" 200 None 2026-05-05 00:30:03,131 - lava_callback - DEBUG - Processing node 69f696fb1f903fec0b0dbd52: baseline-arm-clabbe 2026-05-05 00:30:03,131 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f696fb1f903fec0b0dbd52 2026-05-05 00:30:03,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:30:03,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f696fb1f903fec0b0dbd52 HTTP/1.1" 200 None 2026-05-05 00:30:03,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:30:03,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:30:03,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:30:03,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f696fb1f903fec0b0dbd52 HTTP/1.1" 200 None 2026-05-05 00:30:03,194 - lava_callback - ERROR - Error processing callback for node 69f696fb1f903fec0b0dbd52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52380 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:31:23,227 - lava_callback - INFO - Received callback for node 69f697451f903fec0b0dbd80 INFO: 172.20.0.1:60004 - "POST /node/69f697451f903fec0b0dbd80 HTTP/1.0" 202 Accepted 2026-05-05 00:31:23,233 - lava_callback - INFO - Starting async job submit for node 69f697451f903fec0b0dbd80 2026-05-05 00:31:23,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:31:23,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f697451f903fec0b0dbd80 HTTP/1.1" 200 None 2026-05-05 00:31:23,248 - lava_callback - DEBUG - Processing node 69f697451f903fec0b0dbd80: baseline-arm-clabbe 2026-05-05 00:31:23,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f697451f903fec0b0dbd80 2026-05-05 00:31:23,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:31:23,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f697451f903fec0b0dbd80 HTTP/1.1" 200 None 2026-05-05 00:31:23,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:31:23,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:31:23,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:31:23,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f697451f903fec0b0dbd80 HTTP/1.1" 200 None 2026-05-05 00:31:23,298 - lava_callback - ERROR - Error processing callback for node 69f697451f903fec0b0dbd80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56584 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:32:29,340 - lava_callback - INFO - Received callback for node 69f93a7cd09fe59064f0546f INFO: 172.20.0.1:56588 - "POST /node/69f93a7cd09fe59064f0546f HTTP/1.0" 202 Accepted 2026-05-05 00:32:29,346 - lava_callback - INFO - Starting async job submit for node 69f93a7cd09fe59064f0546f 2026-05-05 00:32:29,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:29,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7cd09fe59064f0546f HTTP/1.1" 200 None 2026-05-05 00:32:29,378 - lava_callback - DEBUG - Processing node 69f93a7cd09fe59064f0546f: baseline-x86 2026-05-05 00:32:29,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:32:29,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:32:29,666 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7cd09fe59064f0546f/log.txt.gz 2026-05-05 00:32:29,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:32:30,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:32:30,084 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7cd09fe59064f0546f/lava-logs.yaml 2026-05-05 00:32:30,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:32:30,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:32:30,230 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7cd09fe59064f0546f/lava_callback.json.gz 2026-05-05 00:32:30,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:30,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7cd09fe59064f0546f HTTP/1.1" 200 None 2026-05-05 00:32:30,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:30,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9372ed09fe59064f02d1d HTTP/1.1" 200 None 2026-05-05 00:32:30,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:30,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93a7cd09fe59064f0546f HTTP/1.1" 200 None 2026-05-05 00:32:30,303 - lava_callback - INFO - Completed processing callback for node 69f93a7cd09fe59064f0546f INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:32:37,480 - lava_callback - INFO - Received callback for node 69f93a7ed09fe59064f05478 INFO: 172.20.0.1:42042 - "POST /node/69f93a7ed09fe59064f05478 HTTP/1.0" 202 Accepted 2026-05-05 00:32:37,486 - lava_callback - INFO - Starting async job submit for node 69f93a7ed09fe59064f05478 2026-05-05 00:32:37,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:37,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7ed09fe59064f05478 HTTP/1.1" 200 None 2026-05-05 00:32:37,518 - lava_callback - DEBUG - Processing node 69f93a7ed09fe59064f05478: baseline-x86 2026-05-05 00:32:37,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:32:37,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:32:37,810 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7ed09fe59064f05478/log.txt.gz 2026-05-05 00:32:37,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:32:38,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:32:38,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7ed09fe59064f05478/lava-logs.yaml 2026-05-05 00:32:38,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:32:38,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:32:38,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7ed09fe59064f05478/lava_callback.json.gz 2026-05-05 00:32:38,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:38,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7ed09fe59064f05478 HTTP/1.1" 200 None 2026-05-05 00:32:38,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:38,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9372ed09fe59064f02d1d HTTP/1.1" 200 None 2026-05-05 00:32:38,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:38,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93a7ed09fe59064f05478 HTTP/1.1" 200 None 2026-05-05 00:32:38,263 - lava_callback - INFO - Completed processing callback for node 69f93a7ed09fe59064f05478 2026-05-05 00:32:43,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:32:43,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59862 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:33:03,282 - lava_callback - INFO - Received callback for node 69f697b31f903fec0b0dbdbb INFO: 172.20.0.1:43184 - "POST /node/69f697b31f903fec0b0dbdbb HTTP/1.0" 202 Accepted 2026-05-05 00:33:03,287 - lava_callback - INFO - Starting async job submit for node 69f697b31f903fec0b0dbdbb 2026-05-05 00:33:03,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:33:03,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f697b31f903fec0b0dbdbb HTTP/1.1" 200 None 2026-05-05 00:33:03,303 - lava_callback - DEBUG - Processing node 69f697b31f903fec0b0dbdbb: baseline-arm-clabbe 2026-05-05 00:33:03,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f697b31f903fec0b0dbdbb 2026-05-05 00:33:03,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:33:03,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f697b31f903fec0b0dbdbb HTTP/1.1" 200 None 2026-05-05 00:33:03,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:33:03,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:33:03,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:33:03,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f697b31f903fec0b0dbdbb HTTP/1.1" 200 None 2026-05-05 00:33:03,353 - lava_callback - ERROR - Error processing callback for node 69f697b31f903fec0b0dbdbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51366 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:34:23,342 - lava_callback - INFO - Received callback for node 69f697ff1f903fec0b0de4c1 INFO: 172.20.0.1:47650 - "POST /node/69f697ff1f903fec0b0de4c1 HTTP/1.0" 202 Accepted 2026-05-05 00:34:23,347 - lava_callback - INFO - Starting async job submit for node 69f697ff1f903fec0b0de4c1 2026-05-05 00:34:23,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:34:23,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f697ff1f903fec0b0de4c1 HTTP/1.1" 200 None 2026-05-05 00:34:23,364 - lava_callback - DEBUG - Processing node 69f697ff1f903fec0b0de4c1: baseline-arm-clabbe 2026-05-05 00:34:23,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f697ff1f903fec0b0de4c1 2026-05-05 00:34:23,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:34:23,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f697ff1f903fec0b0de4c1 HTTP/1.1" 200 None 2026-05-05 00:34:23,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:34:23,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:34:23,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:34:23,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f697ff1f903fec0b0de4c1 HTTP/1.1" 200 None 2026-05-05 00:34:23,417 - lava_callback - ERROR - Error processing callback for node 69f697ff1f903fec0b0de4c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51856 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:35:03,249 - lava_callback - INFO - Received callback for node 69f698251f903fec0b0de4d3 INFO: 172.20.0.1:44014 - "POST /node/69f698251f903fec0b0de4d3 HTTP/1.0" 202 Accepted 2026-05-05 00:35:03,254 - lava_callback - INFO - Starting async job submit for node 69f698251f903fec0b0de4d3 2026-05-05 00:35:03,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:03,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f698251f903fec0b0de4d3 HTTP/1.1" 200 None 2026-05-05 00:35:03,269 - lava_callback - DEBUG - Processing node 69f698251f903fec0b0de4d3: baseline-arm-clabbe 2026-05-05 00:35:03,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f698251f903fec0b0de4d3 2026-05-05 00:35:03,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:03,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f698251f903fec0b0de4d3 HTTP/1.1" 200 None 2026-05-05 00:35:03,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:03,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:35:03,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:03,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f698251f903fec0b0de4d3 HTTP/1.1" 200 None 2026-05-05 00:35:03,314 - lava_callback - ERROR - Error processing callback for node 69f698251f903fec0b0de4d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43616 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:35:43,353 - lava_callback - INFO - Received callback for node 69f6984a1f903fec0b0de4e4 INFO: 172.20.0.1:58180 - "POST /node/69f6984a1f903fec0b0de4e4 HTTP/1.0" 202 Accepted 2026-05-05 00:35:43,358 - lava_callback - INFO - Starting async job submit for node 69f6984a1f903fec0b0de4e4 2026-05-05 00:35:43,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:43,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6984a1f903fec0b0de4e4 HTTP/1.1" 200 None 2026-05-05 00:35:43,373 - lava_callback - DEBUG - Processing node 69f6984a1f903fec0b0de4e4: baseline-arm-clabbe 2026-05-05 00:35:43,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6984a1f903fec0b0de4e4 2026-05-05 00:35:43,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:43,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6984a1f903fec0b0de4e4 HTTP/1.1" 200 None 2026-05-05 00:35:43,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:43,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:35:43,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:35:43,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6984a1f903fec0b0de4e4 HTTP/1.1" 200 None 2026-05-05 00:35:43,428 - lava_callback - ERROR - Error processing callback for node 69f6984a1f903fec0b0de4e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49720 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:39:43,510 - lava_callback - INFO - Received callback for node 69f699481f903fec0b0de57a INFO: 172.20.0.1:59230 - "POST /node/69f699481f903fec0b0de57a HTTP/1.0" 202 Accepted 2026-05-05 00:39:43,518 - lava_callback - INFO - Starting async job submit for node 69f699481f903fec0b0de57a 2026-05-05 00:39:43,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:39:43,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f699481f903fec0b0de57a HTTP/1.1" 200 None 2026-05-05 00:39:43,537 - lava_callback - DEBUG - Processing node 69f699481f903fec0b0de57a: baseline-arm-clabbe 2026-05-05 00:39:43,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f699481f903fec0b0de57a 2026-05-05 00:39:43,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:39:43,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f699481f903fec0b0de57a HTTP/1.1" 200 None 2026-05-05 00:39:43,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:39:43,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 00:39:43,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:39:43,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f699481f903fec0b0de57a HTTP/1.1" 200 None 2026-05-05 00:39:43,597 - lava_callback - ERROR - Error processing callback for node 69f699481f903fec0b0de57a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54084 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:40:22,913 - lava_callback - INFO - Received callback for node 69f93c68d09fe59064f0550e INFO: 172.20.0.1:55996 - "POST /node/69f93c68d09fe59064f0550e HTTP/1.0" 202 Accepted 2026-05-05 00:40:22,919 - lava_callback - INFO - Starting async job submit for node 69f93c68d09fe59064f0550e 2026-05-05 00:40:22,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:22,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c68d09fe59064f0550e HTTP/1.1" 200 None 2026-05-05 00:40:22,937 - lava_callback - DEBUG - Processing node 69f93c68d09fe59064f0550e: baseline-arm 2026-05-05 00:40:22,942 - lava_callback - WARNING - Failed to upload log 2026-05-05 00:40:22,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:23,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:23,109 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c68d09fe59064f0550e/lava-logs.yaml 2026-05-05 00:40:23,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:23,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:23,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c68d09fe59064f0550e/lava_callback.json.gz 2026-05-05 00:40:23,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:23,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c68d09fe59064f0550e HTTP/1.1" 200 None 2026-05-05 00:40:23,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:23,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:40:23,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:23,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c68d09fe59064f0550e HTTP/1.1" 200 None 2026-05-05 00:40:23,277 - lava_callback - INFO - Completed processing callback for node 69f93c68d09fe59064f0550e 2026-05-05 00:40:25,093 - lava_callback - INFO - Received callback for node 69f6996d1f903fec0b0de5a6 INFO: 172.20.0.1:56010 - "POST /node/69f6996d1f903fec0b0de5a6 HTTP/1.0" 202 Accepted 2026-05-05 00:40:25,098 - lava_callback - INFO - Starting async job submit for node 69f6996d1f903fec0b0de5a6 2026-05-05 00:40:25,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:25,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6996d1f903fec0b0de5a6 HTTP/1.1" 200 None 2026-05-05 00:40:25,115 - lava_callback - DEBUG - Processing node 69f6996d1f903fec0b0de5a6: baseline-arm-baylibre 2026-05-05 00:40:25,115 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6996d1f903fec0b0de5a6 2026-05-05 00:40:25,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:25,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6996d1f903fec0b0de5a6 HTTP/1.1" 200 None 2026-05-05 00:40:25,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:25,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 00:40:25,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:25,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6996d1f903fec0b0de5a6 HTTP/1.1" 200 None 2026-05-05 00:40:25,176 - lava_callback - ERROR - Error processing callback for node 69f6996d1f903fec0b0de5a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:40:43,002 - lava_callback - INFO - Received callback for node 69f93c6bd09fe59064f0551a INFO: 172.20.0.1:45852 - "POST /node/69f93c6bd09fe59064f0551a HTTP/1.0" 202 Accepted 2026-05-05 00:40:43,050 - lava_callback - INFO - Starting async job submit for node 69f93c6bd09fe59064f0551a 2026-05-05 00:40:43,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:43,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:43,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 00:40:43,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c6bd09fe59064f0551a HTTP/1.1" 200 None 2026-05-05 00:40:43,521 - lava_callback - DEBUG - Processing node 69f93c6bd09fe59064f0551a: baseline-arm 2026-05-05 00:40:43,666 - lava_callback - INFO - Received callback for node 69f699771f903fec0b0de5c5 2026-05-05 00:40:43,687 - lava_callback - INFO - Starting async job submit for node 69f699771f903fec0b0de5c5 INFO: 172.20.0.1:45868 - "POST /node/69f699771f903fec0b0de5c5 HTTP/1.0" 202 Accepted 2026-05-05 00:40:43,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:43,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:43,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f699771f903fec0b0de5c5 HTTP/1.1" 200 None 2026-05-05 00:40:43,716 - lava_callback - DEBUG - Processing node 69f699771f903fec0b0de5c5: baseline-arm-clabbe 2026-05-05 00:40:43,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f699771f903fec0b0de5c5 2026-05-05 00:40:43,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:43,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f699771f903fec0b0de5c5 HTTP/1.1" 200 None 2026-05-05 00:40:43,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:43,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 00:40:43,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:43,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f699771f903fec0b0de5c5 HTTP/1.1" 200 None 2026-05-05 00:40:43,768 - lava_callback - ERROR - Error processing callback for node 69f699771f903fec0b0de5c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:40:43,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:43,802 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c6bd09fe59064f0551a/log.txt.gz 2026-05-05 00:40:43,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:44,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:44,082 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c6bd09fe59064f0551a/lava-logs.yaml 2026-05-05 00:40:44,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:44,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:44,168 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c6bd09fe59064f0551a/lava_callback.json.gz 2026-05-05 00:40:44,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:44,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c6bd09fe59064f0551a HTTP/1.1" 200 None 2026-05-05 00:40:44,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:44,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:40:44,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:44,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c6bd09fe59064f0551a HTTP/1.1" 200 None 2026-05-05 00:40:44,283 - lava_callback - INFO - Completed processing callback for node 69f93c6bd09fe59064f0551a 2026-05-05 00:40:44,367 - lava_callback - INFO - Received callback for node 69f6997a1f903fec0b0de5fd INFO: 172.20.0.1:45870 - "POST /node/69f6997a1f903fec0b0de5fd HTTP/1.0" 202 Accepted 2026-05-05 00:40:44,373 - lava_callback - INFO - Starting async job submit for node 69f6997a1f903fec0b0de5fd 2026-05-05 00:40:44,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:44,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997a1f903fec0b0de5fd HTTP/1.1" 200 None 2026-05-05 00:40:44,391 - lava_callback - DEBUG - Processing node 69f6997a1f903fec0b0de5fd: baseline-arm-clabbe 2026-05-05 00:40:44,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6997a1f903fec0b0de5fd 2026-05-05 00:40:44,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:44,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997a1f903fec0b0de5fd HTTP/1.1" 200 None 2026-05-05 00:40:44,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:44,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 00:40:44,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:44,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6997a1f903fec0b0de5fd HTTP/1.1" 200 None 2026-05-05 00:40:44,501 - lava_callback - ERROR - Error processing callback for node 69f6997a1f903fec0b0de5fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:40:45,119 - lava_callback - INFO - Received callback for node 69f6997b1f903fec0b0de600 INFO: 172.20.0.1:45872 - "POST /node/69f6997b1f903fec0b0de600 HTTP/1.0" 202 Accepted 2026-05-05 00:40:45,126 - lava_callback - INFO - Starting async job submit for node 69f6997b1f903fec0b0de600 2026-05-05 00:40:45,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997b1f903fec0b0de600 HTTP/1.1" 200 None 2026-05-05 00:40:45,152 - lava_callback - DEBUG - Processing node 69f6997b1f903fec0b0de600: baseline-arm-clabbe 2026-05-05 00:40:45,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6997b1f903fec0b0de600 2026-05-05 00:40:45,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997b1f903fec0b0de600 HTTP/1.1" 200 None 2026-05-05 00:40:45,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 00:40:45,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6997b1f903fec0b0de600 HTTP/1.1" 200 None 2026-05-05 00:40:45,203 - lava_callback - ERROR - Error processing callback for node 69f6997b1f903fec0b0de600: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:40:45,834 - lava_callback - INFO - Received callback for node 69f6997c1f903fec0b0de603 INFO: 172.20.0.1:45876 - "POST /node/69f6997c1f903fec0b0de603 HTTP/1.0" 202 Accepted 2026-05-05 00:40:45,839 - lava_callback - INFO - Starting async job submit for node 69f6997c1f903fec0b0de603 2026-05-05 00:40:45,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997c1f903fec0b0de603 HTTP/1.1" 200 None 2026-05-05 00:40:45,854 - lava_callback - DEBUG - Processing node 69f6997c1f903fec0b0de603: baseline-arm-clabbe 2026-05-05 00:40:45,854 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6997c1f903fec0b0de603 2026-05-05 00:40:45,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997c1f903fec0b0de603 HTTP/1.1" 200 None 2026-05-05 00:40:45,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 00:40:45,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:45,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6997c1f903fec0b0de603 HTTP/1.1" 200 None 2026-05-05 00:40:45,900 - lava_callback - ERROR - Error processing callback for node 69f6997c1f903fec0b0de603: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45886 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:40:46,514 - lava_callback - INFO - Received callback for node 69f6997e1f903fec0b0de60e INFO: 172.20.0.1:45890 - "POST /node/69f6997e1f903fec0b0de60e HTTP/1.0" 202 Accepted 2026-05-05 00:40:46,520 - lava_callback - INFO - Starting async job submit for node 69f6997e1f903fec0b0de60e 2026-05-05 00:40:46,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:46,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997e1f903fec0b0de60e HTTP/1.1" 200 None 2026-05-05 00:40:46,545 - lava_callback - DEBUG - Processing node 69f6997e1f903fec0b0de60e: baseline-arm-clabbe 2026-05-05 00:40:46,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6997e1f903fec0b0de60e 2026-05-05 00:40:46,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:46,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6997e1f903fec0b0de60e HTTP/1.1" 200 None 2026-05-05 00:40:46,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:46,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 00:40:46,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:46,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6997e1f903fec0b0de60e HTTP/1.1" 200 None 2026-05-05 00:40:46,590 - lava_callback - ERROR - Error processing callback for node 69f6997e1f903fec0b0de60e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:40:53,224 - lava_callback - INFO - Received callback for node 69f93c68d09fe59064f05511 INFO: 172.20.0.1:53324 - "POST /node/69f93c68d09fe59064f05511 HTTP/1.0" 202 Accepted 2026-05-05 00:40:53,278 - lava_callback - INFO - Starting async job submit for node 69f93c68d09fe59064f05511 2026-05-05 00:40:53,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:53,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c68d09fe59064f05511 HTTP/1.1" 200 None 2026-05-05 00:40:53,340 - lava_callback - DEBUG - Processing node 69f93c68d09fe59064f05511: baseline-arm 2026-05-05 00:40:53,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:53,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:53,550 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c68d09fe59064f05511/log.txt.gz 2026-05-05 00:40:53,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:53,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:53,797 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c68d09fe59064f05511/lava-logs.yaml 2026-05-05 00:40:53,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:40:53,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:40:53,889 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c68d09fe59064f05511/lava_callback.json.gz 2026-05-05 00:40:53,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:53,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c68d09fe59064f05511 HTTP/1.1" 200 None 2026-05-05 00:40:53,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:40:53,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:40:53,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c68d09fe59064f05511 HTTP/1.1" 200 None 2026-05-05 00:40:53,968 - lava_callback - INFO - Completed processing callback for node 69f93c68d09fe59064f05511 INFO: 172.20.0.1:53340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:41:10,281 - lava_callback - INFO - Received callback for node 69f93a7dd09fe59064f05472 INFO: 172.20.0.1:48656 - "POST /node/69f93a7dd09fe59064f05472 HTTP/1.0" 202 Accepted 2026-05-05 00:41:10,287 - lava_callback - INFO - Starting async job submit for node 69f93a7dd09fe59064f05472 2026-05-05 00:41:10,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:10,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7dd09fe59064f05472 HTTP/1.1" 200 None 2026-05-05 00:41:10,364 - lava_callback - DEBUG - Processing node 69f93a7dd09fe59064f05472: baseline-x86 2026-05-05 00:41:10,443 - lava_callback - INFO - Received callback for node 69f93c6ad09fe59064f05517 INFO: 172.20.0.1:48660 - "POST /node/69f93c6ad09fe59064f05517 HTTP/1.0" 202 Accepted 2026-05-05 00:41:10,465 - lava_callback - INFO - Starting async job submit for node 69f93c6ad09fe59064f05517 2026-05-05 00:41:10,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:10,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c6ad09fe59064f05517 HTTP/1.1" 200 None 2026-05-05 00:41:10,619 - lava_callback - DEBUG - Processing node 69f93c6ad09fe59064f05517: baseline-arm 2026-05-05 00:41:10,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:11,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:11,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:11,073 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7dd09fe59064f05472/log.txt.gz 2026-05-05 00:41:11,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:11,249 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c6ad09fe59064f05517/log.txt.gz 2026-05-05 00:41:11,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:11,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:11,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:11,652 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c6ad09fe59064f05517/lava-logs.yaml 2026-05-05 00:41:11,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:11,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7dd09fe59064f05472/lava-logs.yaml 2026-05-05 00:41:11,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:11,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:11,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:11,860 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c6ad09fe59064f05517/lava_callback.json.gz 2026-05-05 00:41:11,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:11,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:11,909 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7dd09fe59064f05472/lava_callback.json.gz 2026-05-05 00:41:11,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:11,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c6ad09fe59064f05517 HTTP/1.1" 200 None 2026-05-05 00:41:11,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:11,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7dd09fe59064f05472 HTTP/1.1" 200 None 2026-05-05 00:41:11,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:11,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:41:11,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:11,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9372ed09fe59064f02d1d HTTP/1.1" 200 None 2026-05-05 00:41:11,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:12,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c6ad09fe59064f05517 HTTP/1.1" 200 None 2026-05-05 00:41:12,008 - lava_callback - INFO - Completed processing callback for node 69f93c6ad09fe59064f05517 2026-05-05 00:41:12,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93a7dd09fe59064f05472 HTTP/1.1" 200 None 2026-05-05 00:41:12,023 - lava_callback - INFO - Completed processing callback for node 69f93a7dd09fe59064f05472 2026-05-05 00:41:13,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:13,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:48670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:41:42,524 - lava_callback - INFO - Received callback for node 69f93c67d09fe59064f0550b INFO: 172.20.0.1:47008 - "POST /node/69f93c67d09fe59064f0550b HTTP/1.0" 202 Accepted 2026-05-05 00:41:42,530 - lava_callback - INFO - Starting async job submit for node 69f93c67d09fe59064f0550b 2026-05-05 00:41:42,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:42,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c67d09fe59064f0550b HTTP/1.1" 200 None 2026-05-05 00:41:42,567 - lava_callback - DEBUG - Processing node 69f93c67d09fe59064f0550b: baseline-arm 2026-05-05 00:41:42,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:42,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:42,878 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c67d09fe59064f0550b/log.txt.gz 2026-05-05 00:41:42,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:43,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:43,130 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c67d09fe59064f0550b/lava-logs.yaml 2026-05-05 00:41:43,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:43,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:43,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c67d09fe59064f0550b/lava_callback.json.gz 2026-05-05 00:41:43,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:43,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c67d09fe59064f0550b HTTP/1.1" 200 None 2026-05-05 00:41:43,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:43,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:41:43,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:43,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c67d09fe59064f0550b HTTP/1.1" 200 None 2026-05-05 00:41:43,339 - lava_callback - INFO - Completed processing callback for node 69f93c67d09fe59064f0550b 2026-05-05 00:41:43,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:43,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 00:41:44,433 - lava_callback - INFO - Received callback for node 69f93c78d09fe59064f0554f INFO: 172.20.0.1:47018 - "POST /node/69f93c78d09fe59064f0554f HTTP/1.0" 202 Accepted 2026-05-05 00:41:44,438 - lava_callback - INFO - Starting async job submit for node 69f93c78d09fe59064f0554f 2026-05-05 00:41:44,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:44,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c78d09fe59064f0554f HTTP/1.1" 200 None 2026-05-05 00:41:44,475 - lava_callback - DEBUG - Processing node 69f93c78d09fe59064f0554f: baseline-arm-broonie 2026-05-05 00:41:44,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:44,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:44,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c78d09fe59064f0554f/log.txt.gz 2026-05-05 00:41:44,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:45,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:45,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c78d09fe59064f0554f/lava-logs.yaml 2026-05-05 00:41:45,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:45,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:45,162 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c78d09fe59064f0554f/lava_callback.json.gz 2026-05-05 00:41:45,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:45,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c78d09fe59064f0554f HTTP/1.1" 200 None 2026-05-05 00:41:45,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:45,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:41:45,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:45,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c78d09fe59064f0554f HTTP/1.1" 200 None 2026-05-05 00:41:45,236 - lava_callback - INFO - Completed processing callback for node 69f93c78d09fe59064f0554f 2026-05-05 00:41:46,052 - lava_callback - INFO - Received callback for node 69f93c79d09fe59064f05554 INFO: 172.20.0.1:47026 - "POST /node/69f93c79d09fe59064f05554 HTTP/1.0" 202 Accepted 2026-05-05 00:41:46,058 - lava_callback - INFO - Starting async job submit for node 69f93c79d09fe59064f05554 2026-05-05 00:41:46,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:46,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c79d09fe59064f05554 HTTP/1.1" 200 None 2026-05-05 00:41:46,095 - lava_callback - DEBUG - Processing node 69f93c79d09fe59064f05554: baseline-arm-broonie 2026-05-05 00:41:46,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:46,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:46,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c79d09fe59064f05554/log.txt.gz INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:41:46,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:46,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:46,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c79d09fe59064f05554/lava-logs.yaml 2026-05-05 00:41:46,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:46,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:46,785 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c79d09fe59064f05554/lava_callback.json.gz 2026-05-05 00:41:46,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:46,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c79d09fe59064f05554 HTTP/1.1" 200 None 2026-05-05 00:41:46,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:46,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:41:46,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:46,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c79d09fe59064f05554 HTTP/1.1" 200 None 2026-05-05 00:41:46,857 - lava_callback - INFO - Completed processing callback for node 69f93c79d09fe59064f05554 2026-05-05 00:41:50,359 - lava_callback - INFO - Received callback for node 69f93c8bd09fe59064f05597 INFO: 172.20.0.1:39728 - "POST /node/69f93c8bd09fe59064f05597 HTTP/1.0" 202 Accepted 2026-05-05 00:41:50,405 - lava_callback - INFO - Starting async job submit for node 69f93c8bd09fe59064f05597 2026-05-05 00:41:50,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:50,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c8bd09fe59064f05597 HTTP/1.1" 200 None 2026-05-05 00:41:50,439 - lava_callback - DEBUG - Processing node 69f93c8bd09fe59064f05597: baseline-arm-pengutronix 2026-05-05 00:41:50,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:50,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:50,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c8bd09fe59064f05597/log.txt.gz 2026-05-05 00:41:50,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:50,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:50,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c8bd09fe59064f05597/lava-logs.yaml 2026-05-05 00:41:50,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:50,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:50,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c8bd09fe59064f05597/lava_callback.json.gz 2026-05-05 00:41:50,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:50,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c8bd09fe59064f05597 HTTP/1.1" 200 None 2026-05-05 00:41:50,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:50,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:41:50,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:50,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c8bd09fe59064f05597 HTTP/1.1" 200 None 2026-05-05 00:41:50,961 - lava_callback - INFO - Completed processing callback for node 69f93c8bd09fe59064f05597 INFO: 172.20.0.1:39734 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:41:57,886 - lava_callback - INFO - Received callback for node 69f93c88d09fe59064f05591 INFO: 172.20.0.1:39746 - "POST /node/69f93c88d09fe59064f05591 HTTP/1.0" 202 Accepted 2026-05-05 00:41:57,892 - lava_callback - INFO - Starting async job submit for node 69f93c88d09fe59064f05591 2026-05-05 00:41:57,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:57,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c88d09fe59064f05591 HTTP/1.1" 200 None 2026-05-05 00:41:57,924 - lava_callback - DEBUG - Processing node 69f93c88d09fe59064f05591: baseline-arm-pengutronix 2026-05-05 00:41:58,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:58,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:58,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c88d09fe59064f05591/log.txt.gz 2026-05-05 00:41:58,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:58,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:58,423 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c88d09fe59064f05591/lava-logs.yaml 2026-05-05 00:41:58,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:41:58,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:41:58,519 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c88d09fe59064f05591/lava_callback.json.gz 2026-05-05 00:41:58,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:58,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c88d09fe59064f05591 HTTP/1.1" 200 None 2026-05-05 00:41:58,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:58,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:41:58,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:41:58,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c88d09fe59064f05591 HTTP/1.1" 200 None 2026-05-05 00:41:58,611 - lava_callback - INFO - Completed processing callback for node 69f93c88d09fe59064f05591 2026-05-05 00:42:02,885 - lava_callback - INFO - Received callback for node 69f93cc2d09fe59064f0563f INFO: 172.20.0.1:39310 - "POST /node/69f93cc2d09fe59064f0563f HTTP/1.0" 202 Accepted 2026-05-05 00:42:02,891 - lava_callback - INFO - Starting async job submit for node 69f93cc2d09fe59064f0563f 2026-05-05 00:42:02,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:02,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cc2d09fe59064f0563f HTTP/1.1" 200 None 2026-05-05 00:42:02,908 - lava_callback - DEBUG - Processing node 69f93cc2d09fe59064f0563f: baseline-arm 2026-05-05 00:42:02,913 - lava_callback - WARNING - Failed to upload log 2026-05-05 00:42:02,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:02,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:02,995 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93cc2d09fe59064f0563f/lava-logs.yaml 2026-05-05 00:42:02,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:03,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:03,087 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93cc2d09fe59064f0563f/lava_callback.json.gz 2026-05-05 00:42:03,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:03,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cc2d09fe59064f0563f HTTP/1.1" 200 None 2026-05-05 00:42:03,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:03,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:42:03,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:03,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93cc2d09fe59064f0563f HTTP/1.1" 200 None 2026-05-05 00:42:03,152 - lava_callback - INFO - Completed processing callback for node 69f93cc2d09fe59064f0563f INFO: 172.20.0.1:39324 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:42:13,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:13,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:42134 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 00:42:15,489 - lava_callback - INFO - Received callback for node 69f93c7ed09fe59064f0556a INFO: 172.20.0.1:42150 - "POST /node/69f93c7ed09fe59064f0556a HTTP/1.0" 202 Accepted 2026-05-05 00:42:15,495 - lava_callback - INFO - Starting async job submit for node 69f93c7ed09fe59064f0556a 2026-05-05 00:42:15,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:15,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c7ed09fe59064f0556a HTTP/1.1" 200 None 2026-05-05 00:42:15,560 - lava_callback - DEBUG - Processing node 69f93c7ed09fe59064f0556a: baseline-arm-clabbe 2026-05-05 00:42:15,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:15,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:15,889 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c7ed09fe59064f0556a/log.txt.gz 2026-05-05 00:42:16,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:16,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:16,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c7ed09fe59064f0556a/lava-logs.yaml 2026-05-05 00:42:16,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:16,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:16,247 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c7ed09fe59064f0556a/lava_callback.json.gz 2026-05-05 00:42:16,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:16,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c7ed09fe59064f0556a HTTP/1.1" 200 None 2026-05-05 00:42:16,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:16,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:42:16,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:16,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c7ed09fe59064f0556a HTTP/1.1" 200 None 2026-05-05 00:42:16,324 - lava_callback - INFO - Completed processing callback for node 69f93c7ed09fe59064f0556a INFO: 172.20.0.1:42156 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:42:22,094 - lava_callback - INFO - Received callback for node 69f93c85d09fe59064f05583 INFO: 172.20.0.1:35682 - "POST /node/69f93c85d09fe59064f05583 HTTP/1.0" 202 Accepted 2026-05-05 00:42:22,100 - lava_callback - INFO - Starting async job submit for node 69f93c85d09fe59064f05583 2026-05-05 00:42:22,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:22,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c85d09fe59064f05583 HTTP/1.1" 200 None 2026-05-05 00:42:22,159 - lava_callback - DEBUG - Processing node 69f93c85d09fe59064f05583: baseline-arm-kontron 2026-05-05 00:42:22,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:22,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:22,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f93c85d09fe59064f05583/log.txt.gz 2026-05-05 00:42:22,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:22,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:22,711 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f93c85d09fe59064f05583/lava-logs.yaml 2026-05-05 00:42:22,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:22,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:22,816 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f93c85d09fe59064f05583/lava_callback.json.gz 2026-05-05 00:42:22,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:22,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c85d09fe59064f05583 HTTP/1.1" 200 None 2026-05-05 00:42:22,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:22,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:42:22,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:22,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c85d09fe59064f05583 HTTP/1.1" 200 None 2026-05-05 00:42:22,899 - lava_callback - INFO - Completed processing callback for node 69f93c85d09fe59064f05583 2026-05-05 00:42:23,177 - lava_callback - INFO - Received callback for node 69f93c86d09fe59064f05586 INFO: 172.20.0.1:35696 - "POST /node/69f93c86d09fe59064f05586 HTTP/1.0" 202 Accepted 2026-05-05 00:42:23,183 - lava_callback - INFO - Starting async job submit for node 69f93c86d09fe59064f05586 2026-05-05 00:42:23,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:23,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c86d09fe59064f05586 HTTP/1.1" 200 None 2026-05-05 00:42:23,221 - lava_callback - DEBUG - Processing node 69f93c86d09fe59064f05586: baseline-arm-kontron 2026-05-05 00:42:23,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:23,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:23,511 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f93c86d09fe59064f05586/log.txt.gz 2026-05-05 00:42:23,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:23,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:23,953 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f93c86d09fe59064f05586/lava-logs.yaml 2026-05-05 00:42:23,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:24,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:24,067 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69f93c86d09fe59064f05586/lava_callback.json.gz 2026-05-05 00:42:24,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:24,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c86d09fe59064f05586 HTTP/1.1" 200 None 2026-05-05 00:42:24,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:24,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:42:24,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:24,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c86d09fe59064f05586 HTTP/1.1" 200 None 2026-05-05 00:42:24,142 - lava_callback - INFO - Completed processing callback for node 69f93c86d09fe59064f05586 INFO: 172.20.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:42:42,417 - lava_callback - INFO - Received callback for node 69f93ca2d09fe59064f055f1 INFO: 172.20.0.1:42728 - "POST /node/69f93ca2d09fe59064f055f1 HTTP/1.0" 202 Accepted 2026-05-05 00:42:42,423 - lava_callback - INFO - Starting async job submit for node 69f93ca2d09fe59064f055f1 2026-05-05 00:42:42,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:42,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca2d09fe59064f055f1 HTTP/1.1" 200 None 2026-05-05 00:42:42,461 - lava_callback - DEBUG - Processing node 69f93ca2d09fe59064f055f1: ltp-dio 2026-05-05 00:42:42,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:42,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:42,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69f93ca2d09fe59064f055f1/log.txt.gz 2026-05-05 00:42:43,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:43,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:43,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69f93ca2d09fe59064f055f1/lava-logs.yaml 2026-05-05 00:42:43,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:43,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:43,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 00:42:43,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:43,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69f93ca2d09fe59064f055f1/lava_callback.json.gz 2026-05-05 00:42:43,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:43,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca2d09fe59064f055f1 HTTP/1.1" 200 None 2026-05-05 00:42:43,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:43,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:42:43,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:43,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ca2d09fe59064f055f1 HTTP/1.1" 200 None 2026-05-05 00:42:43,607 - lava_callback - INFO - Completed processing callback for node 69f93ca2d09fe59064f055f1 INFO: 172.20.0.1:42738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60476 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:42:58,196 - lava_callback - INFO - Received callback for node 69f93c89d09fe59064f05594 INFO: 172.20.0.1:60478 - "POST /node/69f93c89d09fe59064f05594 HTTP/1.0" 202 Accepted 2026-05-05 00:42:58,202 - lava_callback - INFO - Starting async job submit for node 69f93c89d09fe59064f05594 2026-05-05 00:42:58,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:58,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c89d09fe59064f05594 HTTP/1.1" 200 None 2026-05-05 00:42:58,274 - lava_callback - DEBUG - Processing node 69f93c89d09fe59064f05594: baseline-arm-pengutronix 2026-05-05 00:42:58,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:58,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:58,696 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c89d09fe59064f05594/log.txt.gz 2026-05-05 00:42:58,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:58,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:58,990 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c89d09fe59064f05594/lava-logs.yaml 2026-05-05 00:42:58,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:42:59,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:42:59,173 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93c89d09fe59064f05594/lava_callback.json.gz 2026-05-05 00:42:59,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:59,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c89d09fe59064f05594 HTTP/1.1" 200 None 2026-05-05 00:42:59,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:59,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:42:59,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:42:59,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c89d09fe59064f05594 HTTP/1.1" 200 None 2026-05-05 00:42:59,358 - lava_callback - INFO - Completed processing callback for node 69f93c89d09fe59064f05594 2026-05-05 00:43:05,234 - lava_callback - INFO - Received callback for node 69f69a041f903fec0b0de717 INFO: 172.20.0.1:57198 - "POST /node/69f69a041f903fec0b0de717 HTTP/1.0" 202 Accepted 2026-05-05 00:43:05,239 - lava_callback - INFO - Starting async job submit for node 69f69a041f903fec0b0de717 2026-05-05 00:43:05,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:05,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69a041f903fec0b0de717 HTTP/1.1" 200 None 2026-05-05 00:43:05,254 - lava_callback - DEBUG - Processing node 69f69a041f903fec0b0de717: baseline-arm64-baylibre 2026-05-05 00:43:05,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69a041f903fec0b0de717 2026-05-05 00:43:05,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:05,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69a041f903fec0b0de717 HTTP/1.1" 200 None 2026-05-05 00:43:05,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:05,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 00:43:05,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:05,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69a041f903fec0b0de717 HTTP/1.1" 200 None 2026-05-05 00:43:05,304 - lava_callback - ERROR - Error processing callback for node 69f69a041f903fec0b0de717: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57210 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:43:10,350 - lava_callback - INFO - Received callback for node 69f93c7fd09fe59064f0556e INFO: 172.20.0.1:58268 - "POST /node/69f93c7fd09fe59064f0556e HTTP/1.0" 202 Accepted 2026-05-05 00:43:10,356 - lava_callback - INFO - Starting async job submit for node 69f93c7fd09fe59064f0556e 2026-05-05 00:43:10,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:10,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c7fd09fe59064f0556e HTTP/1.1" 200 None 2026-05-05 00:43:10,398 - lava_callback - DEBUG - Processing node 69f93c7fd09fe59064f0556e: baseline-arm-clabbe 2026-05-05 00:43:10,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:43:10,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:43:10,710 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c7fd09fe59064f0556e/log.txt.gz 2026-05-05 00:43:10,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:43:10,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:43:11,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c7fd09fe59064f0556e/lava-logs.yaml 2026-05-05 00:43:11,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:43:11,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:43:11,109 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c7fd09fe59064f0556e/lava_callback.json.gz 2026-05-05 00:43:11,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:11,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c7fd09fe59064f0556e HTTP/1.1" 200 None 2026-05-05 00:43:11,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:11,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:43:11,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:11,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c7fd09fe59064f0556e HTTP/1.1" 200 None 2026-05-05 00:43:11,182 - lava_callback - INFO - Completed processing callback for node 69f93c7fd09fe59064f0556e 2026-05-05 00:43:13,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:13,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:43:55,418 - lava_callback - INFO - Received callback for node 69f93c83d09fe59064f0557d INFO: 172.20.0.1:47630 - "POST /node/69f93c83d09fe59064f0557d HTTP/1.0" 202 Accepted 2026-05-05 00:43:55,424 - lava_callback - INFO - Starting async job submit for node 69f93c83d09fe59064f0557d 2026-05-05 00:43:55,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:55,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c83d09fe59064f0557d HTTP/1.1" 200 None 2026-05-05 00:43:55,463 - lava_callback - DEBUG - Processing node 69f93c83d09fe59064f0557d: baseline-arm-clabbe 2026-05-05 00:43:55,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:43:55,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:43:55,769 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c83d09fe59064f0557d/log.txt.gz 2026-05-05 00:43:55,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:43:56,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:43:56,024 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c83d09fe59064f0557d/lava-logs.yaml 2026-05-05 00:43:56,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:43:56,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:43:56,124 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69f93c83d09fe59064f0557d/lava_callback.json.gz 2026-05-05 00:43:56,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:56,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c83d09fe59064f0557d HTTP/1.1" 200 None 2026-05-05 00:43:56,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:56,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:43:56,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:43:56,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c83d09fe59064f0557d HTTP/1.1" 200 None 2026-05-05 00:43:56,203 - lava_callback - INFO - Completed processing callback for node 69f93c83d09fe59064f0557d INFO: 172.20.0.1:47632 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:44:04,315 - lava_callback - INFO - Received callback for node 69f93c76d09fe59064f0554a INFO: 172.20.0.1:49554 - "POST /node/69f93c76d09fe59064f0554a HTTP/1.0" 202 Accepted 2026-05-05 00:44:04,321 - lava_callback - INFO - Starting async job submit for node 69f93c76d09fe59064f0554a 2026-05-05 00:44:04,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:04,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c76d09fe59064f0554a HTTP/1.1" 200 None 2026-05-05 00:44:04,357 - lava_callback - DEBUG - Processing node 69f93c76d09fe59064f0554a: baseline-arm-broonie 2026-05-05 00:44:04,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:44:04,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:44:04,812 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c76d09fe59064f0554a/log.txt.gz 2026-05-05 00:44:04,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:44:05,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:44:05,100 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c76d09fe59064f0554a/lava-logs.yaml 2026-05-05 00:44:05,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:44:05,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:44:05,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c76d09fe59064f0554a/lava_callback.json.gz 2026-05-05 00:44:05,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:05,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c76d09fe59064f0554a HTTP/1.1" 200 None 2026-05-05 00:44:05,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:05,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:44:05,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:05,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c76d09fe59064f0554a HTTP/1.1" 200 None 2026-05-05 00:44:05,313 - lava_callback - INFO - Completed processing callback for node 69f93c76d09fe59064f0554a INFO: 172.20.0.1:49560 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:44:13,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:13,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:44:23,692 - lava_callback - INFO - Received callback for node 69f69a541f903fec0b0de78b INFO: 172.20.0.1:57126 - "POST /node/69f69a541f903fec0b0de78b HTTP/1.0" 202 Accepted 2026-05-05 00:44:23,698 - lava_callback - INFO - Starting async job submit for node 69f69a541f903fec0b0de78b 2026-05-05 00:44:23,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:23,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69a541f903fec0b0de78b HTTP/1.1" 200 None 2026-05-05 00:44:23,713 - lava_callback - DEBUG - Processing node 69f69a541f903fec0b0de78b: baseline-arm64-clabbe 2026-05-05 00:44:23,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69a541f903fec0b0de78b 2026-05-05 00:44:23,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:23,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69a541f903fec0b0de78b HTTP/1.1" 200 None 2026-05-05 00:44:23,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:23,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 00:44:23,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:44:23,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69a541f903fec0b0de78b HTTP/1.1" 200 None 2026-05-05 00:44:23,763 - lava_callback - ERROR - Error processing callback for node 69f69a541f903fec0b0de78b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54526 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:45:23,852 - lava_callback - INFO - Received callback for node 69f69aa01f903fec0b0de7d6 INFO: 172.20.0.1:54500 - "POST /node/69f69aa01f903fec0b0de7d6 HTTP/1.0" 202 Accepted 2026-05-05 00:45:23,858 - lava_callback - INFO - Starting async job submit for node 69f69aa01f903fec0b0de7d6 2026-05-05 00:45:23,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:23,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69aa01f903fec0b0de7d6 HTTP/1.1" 200 None 2026-05-05 00:45:23,872 - lava_callback - DEBUG - Processing node 69f69aa01f903fec0b0de7d6: baseline-arm64-clabbe 2026-05-05 00:45:23,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69aa01f903fec0b0de7d6 2026-05-05 00:45:23,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:23,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69aa01f903fec0b0de7d6 HTTP/1.1" 200 None 2026-05-05 00:45:23,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:23,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 00:45:23,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:23,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69aa01f903fec0b0de7d6 HTTP/1.1" 200 None 2026-05-05 00:45:23,929 - lava_callback - ERROR - Error processing callback for node 69f69aa01f903fec0b0de7d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54514 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:45:33,964 - lava_callback - INFO - Received callback for node 69f93c7ad09fe59064f05557 INFO: 172.20.0.1:50642 - "POST /node/69f93c7ad09fe59064f05557 HTTP/1.0" 202 Accepted 2026-05-05 00:45:33,970 - lava_callback - INFO - Starting async job submit for node 69f93c7ad09fe59064f05557 2026-05-05 00:45:34,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:34,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c7ad09fe59064f05557 HTTP/1.1" 200 None 2026-05-05 00:45:34,032 - lava_callback - DEBUG - Processing node 69f93c7ad09fe59064f05557: baseline-arm-broonie 2026-05-05 00:45:34,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:45:34,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:45:34,385 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c7ad09fe59064f05557/log.txt.gz 2026-05-05 00:45:34,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:45:34,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:45:34,853 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c7ad09fe59064f05557/lava-logs.yaml 2026-05-05 00:45:34,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:45:34,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:45:34,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93c7ad09fe59064f05557/lava_callback.json.gz 2026-05-05 00:45:35,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:35,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c7ad09fe59064f05557 HTTP/1.1" 200 None 2026-05-05 00:45:35,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:35,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:45:35,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:35,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c7ad09fe59064f05557 HTTP/1.1" 200 None 2026-05-05 00:45:35,151 - lava_callback - INFO - Completed processing callback for node 69f93c7ad09fe59064f05557 INFO: 172.20.0.1:50648 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:45:43,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:45:43,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32912 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:47:39,383 - lava_callback - INFO - Received callback for node 69f93a7ed09fe59064f05475 INFO: 172.20.0.1:32926 - "POST /node/69f93a7ed09fe59064f05475 HTTP/1.0" 202 Accepted 2026-05-05 00:47:39,390 - lava_callback - INFO - Starting async job submit for node 69f93a7ed09fe59064f05475 2026-05-05 00:47:39,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:47:39,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7ed09fe59064f05475 HTTP/1.1" 200 None 2026-05-05 00:47:39,449 - lava_callback - DEBUG - Processing node 69f93a7ed09fe59064f05475: baseline-x86 2026-05-05 00:47:40,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:47:40,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:47:40,508 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7ed09fe59064f05475/log.txt.gz 2026-05-05 00:47:40,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:47:41,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:47:41,153 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7ed09fe59064f05475/lava-logs.yaml 2026-05-05 00:47:41,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:47:41,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:47:41,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a7ed09fe59064f05475/lava_callback.json.gz DEBUG: Setup failed for 69f93a7ed09fe59064f05475. Transitting job node result: incomplete -> fail 2026-05-05 00:47:41,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:47:41,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a7ed09fe59064f05475 HTTP/1.1" 200 None 2026-05-05 00:47:41,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:47:41,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9372ed09fe59064f02d1d HTTP/1.1" 200 None 2026-05-05 00:47:41,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:47:41,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93a7ed09fe59064f05475 HTTP/1.1" 200 None 2026-05-05 00:47:41,544 - lava_callback - INFO - Completed processing callback for node 69f93a7ed09fe59064f05475 2026-05-05 00:47:43,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:47:43,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39350 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:48:05,271 - lava_callback - INFO - Received callback for node 69f93c9bd09fe59064f055e1 INFO: 172.20.0.1:35060 - "POST /node/69f93c9bd09fe59064f055e1 HTTP/1.0" 202 Accepted 2026-05-05 00:48:05,284 - lava_callback - INFO - Starting async job submit for node 69f93c9bd09fe59064f055e1 2026-05-05 00:48:05,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:48:05,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c9bd09fe59064f055e1 HTTP/1.1" 200 None 2026-05-05 00:48:05,358 - lava_callback - DEBUG - Processing node 69f93c9bd09fe59064f055e1: ltp-cap_bounds 2026-05-05 00:48:05,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:48:06,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:48:06,108 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69f93c9bd09fe59064f055e1/log.txt.gz INFO: 172.20.0.1:35070 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:48:06,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:48:06,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:48:06,851 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69f93c9bd09fe59064f055e1/lava-logs.yaml 2026-05-05 00:48:06,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:48:07,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:48:07,059 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69f93c9bd09fe59064f055e1/lava_callback.json.gz 2026-05-05 00:48:07,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:48:07,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c9bd09fe59064f055e1 HTTP/1.1" 200 None 2026-05-05 00:48:07,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:48:07,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:48:07,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:48:07,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c9bd09fe59064f055e1 HTTP/1.1" 200 None 2026-05-05 00:48:07,197 - lava_callback - INFO - Completed processing callback for node 69f93c9bd09fe59064f055e1 2026-05-05 00:48:13,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:48:13,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48462 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:49:11,961 - lava_callback - INFO - Received callback for node 69f93cafd09fe59064f05633 INFO: 172.20.0.1:43120 - "POST /node/69f93cafd09fe59064f05633 HTTP/1.0" 202 Accepted 2026-05-05 00:49:11,968 - lava_callback - INFO - Starting async job submit for node 69f93cafd09fe59064f05633 2026-05-05 00:49:11,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:12,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cafd09fe59064f05633 HTTP/1.1" 200 None 2026-05-05 00:49:12,007 - lava_callback - DEBUG - Processing node 69f93cafd09fe59064f05633: ltp-syscalls 2026-05-05 00:49:12,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:12,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:12,836 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69f93cafd09fe59064f05633/log.txt.gz 2026-05-05 00:49:13,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:13,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:13,607 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69f93cafd09fe59064f05633/lava-logs.yaml 2026-05-05 00:49:13,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:13,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:13,730 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69f93cafd09fe59064f05633/lava_callback.json.gz 2026-05-05 00:49:13,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:13,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cafd09fe59064f05633 HTTP/1.1" 200 None 2026-05-05 00:49:13,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:13,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:49:13,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93cafd09fe59064f05633 HTTP/1.1" 200 None 2026-05-05 00:49:13,805 - lava_callback - INFO - Completed processing callback for node 69f93cafd09fe59064f05633 INFO: 172.20.0.1:43134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46558 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:49:32,581 - lava_callback - INFO - Received callback for node 69f93c9dd09fe59064f055e4 INFO: 172.20.0.1:45896 - "POST /node/69f93c9dd09fe59064f055e4 HTTP/1.0" 202 Accepted 2026-05-05 00:49:32,588 - lava_callback - INFO - Starting async job submit for node 69f93c9dd09fe59064f055e4 2026-05-05 00:49:32,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:32,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c9dd09fe59064f055e4 HTTP/1.1" 200 None 2026-05-05 00:49:32,626 - lava_callback - DEBUG - Processing node 69f93c9dd09fe59064f055e4: ltp-containers 2026-05-05 00:49:33,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:33,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:33,111 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69f93c9dd09fe59064f055e4/log.txt.gz 2026-05-05 00:49:33,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:33,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:33,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69f93c9dd09fe59064f055e4/lava-logs.yaml 2026-05-05 00:49:33,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:33,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:33,704 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69f93c9dd09fe59064f055e4/lava_callback.json.gz 2026-05-05 00:49:33,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:33,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c9dd09fe59064f055e4 HTTP/1.1" 200 None 2026-05-05 00:49:33,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:33,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:49:33,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:33,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c9dd09fe59064f055e4 HTTP/1.1" 200 None 2026-05-05 00:49:33,787 - lava_callback - INFO - Completed processing callback for node 69f93c9dd09fe59064f055e4 INFO: 172.20.0.1:45910 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:49:43,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:43,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 00:49:43,917 - lava_callback - INFO - Received callback for node 69f69ba11f903fec0b0de8a3 INFO: 172.20.0.1:36074 - "POST /node/69f69ba11f903fec0b0de8a3 HTTP/1.0" 202 Accepted 2026-05-05 00:49:43,922 - lava_callback - INFO - Starting async job submit for node 69f69ba11f903fec0b0de8a3 2026-05-05 00:49:43,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:43,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69ba11f903fec0b0de8a3 HTTP/1.1" 200 None 2026-05-05 00:49:43,938 - lava_callback - DEBUG - Processing node 69f69ba11f903fec0b0de8a3: baseline-arm64-clabbe 2026-05-05 00:49:43,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69ba11f903fec0b0de8a3 2026-05-05 00:49:43,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:43,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69ba11f903fec0b0de8a3 HTTP/1.1" 200 None 2026-05-05 00:49:43,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:43,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 00:49:43,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:43,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69ba11f903fec0b0de8a3 HTTP/1.1" 200 None 2026-05-05 00:49:43,994 - lava_callback - ERROR - Error processing callback for node 69f69ba11f903fec0b0de8a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36076 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:49:52,849 - lava_callback - INFO - Received callback for node 69f93c9ed09fe59064f055e7 INFO: 172.20.0.1:50894 - "POST /node/69f93c9ed09fe59064f055e7 HTTP/1.0" 202 Accepted 2026-05-05 00:49:52,866 - lava_callback - INFO - Starting async job submit for node 69f93c9ed09fe59064f055e7 2026-05-05 00:49:52,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:52,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c9ed09fe59064f055e7 HTTP/1.1" 200 None 2026-05-05 00:49:52,945 - lava_callback - DEBUG - Processing node 69f93c9ed09fe59064f055e7: ltp-controllers 2026-05-05 00:49:53,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:53,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:53,486 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69f93c9ed09fe59064f055e7/log.txt.gz 2026-05-05 00:49:53,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:53,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:53,885 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69f93c9ed09fe59064f055e7/lava-logs.yaml 2026-05-05 00:49:53,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:49:54,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:49:54,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69f93c9ed09fe59064f055e7/lava_callback.json.gz 2026-05-05 00:49:54,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:54,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c9ed09fe59064f055e7 HTTP/1.1" 200 None 2026-05-05 00:49:54,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:54,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:49:54,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:49:54,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c9ed09fe59064f055e7 HTTP/1.1" 200 None 2026-05-05 00:49:54,098 - lava_callback - INFO - Completed processing callback for node 69f93c9ed09fe59064f055e7 INFO: 172.20.0.1:50906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60964 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:50:12,956 - lava_callback - INFO - Received callback for node 69f93eb3d09fe59064f057a7 INFO: 172.20.0.1:39870 - "POST /node/69f93eb3d09fe59064f057a7 HTTP/1.0" 202 Accepted 2026-05-05 00:50:12,962 - lava_callback - INFO - Starting async job submit for node 69f93eb3d09fe59064f057a7 2026-05-05 00:50:12,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:50:12,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93eb3d09fe59064f057a7 HTTP/1.1" 200 None 2026-05-05 00:50:12,980 - lava_callback - DEBUG - Processing node 69f93eb3d09fe59064f057a7: baseline-arm 2026-05-05 00:50:12,985 - lava_callback - WARNING - Failed to upload log 2026-05-05 00:50:12,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:50:13,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:50:13,083 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93eb3d09fe59064f057a7/lava-logs.yaml 2026-05-05 00:50:13,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:50:13,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:50:13,181 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93eb3d09fe59064f057a7/lava_callback.json.gz 2026-05-05 00:50:13,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:50:13,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93eb3d09fe59064f057a7 HTTP/1.1" 200 None 2026-05-05 00:50:13,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:50:13,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:50:13,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:50:13,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93eb3d09fe59064f057a7 HTTP/1.1" 200 None 2026-05-05 00:50:13,239 - lava_callback - INFO - Completed processing callback for node 69f93eb3d09fe59064f057a7 2026-05-05 00:50:13,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:50:13,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49504 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:51:51,159 - lava_callback - INFO - Received callback for node 69f93ee4d09fe59064f057be INFO: 172.20.0.1:45898 - "POST /node/69f93ee4d09fe59064f057be HTTP/1.0" 202 Accepted 2026-05-05 00:51:51,165 - lava_callback - INFO - Starting async job submit for node 69f93ee4d09fe59064f057be 2026-05-05 00:51:51,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:51:51,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ee4d09fe59064f057be HTTP/1.1" 200 None 2026-05-05 00:51:51,203 - lava_callback - DEBUG - Processing node 69f93ee4d09fe59064f057be: baseline-arm-pengutronix 2026-05-05 00:51:51,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:51:51,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:51:51,474 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93ee4d09fe59064f057be/log.txt.gz 2026-05-05 00:51:51,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:51:51,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:51:51,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93ee4d09fe59064f057be/lava-logs.yaml 2026-05-05 00:51:51,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:51:52,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:51:52,312 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69f93ee4d09fe59064f057be/lava_callback.json.gz 2026-05-05 00:51:52,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:51:52,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ee4d09fe59064f057be HTTP/1.1" 200 None 2026-05-05 00:51:52,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:51:52,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:51:52,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:51:52,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ee4d09fe59064f057be HTTP/1.1" 200 None 2026-05-05 00:51:52,383 - lava_callback - INFO - Completed processing callback for node 69f93ee4d09fe59064f057be INFO: 172.20.0.1:45914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:52:13,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:52:13,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43012 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:52:25,973 - lava_callback - INFO - Received callback for node 69f69c471f903fec0b0de91c INFO: 172.20.0.1:34828 - "POST /node/69f69c471f903fec0b0de91c HTTP/1.0" 202 Accepted 2026-05-05 00:52:25,978 - lava_callback - INFO - Starting async job submit for node 69f69c471f903fec0b0de91c 2026-05-05 00:52:25,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:52:25,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c471f903fec0b0de91c HTTP/1.1" 200 None 2026-05-05 00:52:25,993 - lava_callback - DEBUG - Processing node 69f69c471f903fec0b0de91c: baseline-arm-baylibre 2026-05-05 00:52:25,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c471f903fec0b0de91c 2026-05-05 00:52:25,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:52:26,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c471f903fec0b0de91c HTTP/1.1" 200 None 2026-05-05 00:52:26,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:52:26,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 00:52:26,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:52:26,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c471f903fec0b0de91c HTTP/1.1" 200 None 2026-05-05 00:52:26,056 - lava_callback - ERROR - Error processing callback for node 69f69c471f903fec0b0de91c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59930 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:53:05,729 - lava_callback - INFO - Received callback for node 69f69c6c1f903fec0b0de9bc INFO: 172.20.0.1:51156 - "POST /node/69f69c6c1f903fec0b0de9bc HTTP/1.0" 202 Accepted 2026-05-05 00:53:05,737 - lava_callback - INFO - Starting async job submit for node 69f69c6c1f903fec0b0de9bc 2026-05-05 00:53:05,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:05,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c6c1f903fec0b0de9bc HTTP/1.1" 200 None 2026-05-05 00:53:05,797 - lava_callback - DEBUG - Processing node 69f69c6c1f903fec0b0de9bc: baseline-arm-baylibre 2026-05-05 00:53:05,797 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c6c1f903fec0b0de9bc 2026-05-05 00:53:05,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:05,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c6c1f903fec0b0de9bc HTTP/1.1" 200 None 2026-05-05 00:53:05,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:05,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 00:53:05,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:05,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c6c1f903fec0b0de9bc HTTP/1.1" 200 None 2026-05-05 00:53:05,869 - lava_callback - ERROR - Error processing callback for node 69f69c6c1f903fec0b0de9bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:53:06,376 - lava_callback - INFO - Received callback for node 69f69c691f903fec0b0de9b4 INFO: 172.20.0.1:51160 - "POST /node/69f69c691f903fec0b0de9b4 HTTP/1.0" 202 Accepted 2026-05-05 00:53:06,382 - lava_callback - INFO - Starting async job submit for node 69f69c691f903fec0b0de9b4 2026-05-05 00:53:06,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:06,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c691f903fec0b0de9b4 HTTP/1.1" 200 None 2026-05-05 00:53:06,398 - lava_callback - DEBUG - Processing node 69f69c691f903fec0b0de9b4: baseline-arm64-baylibre 2026-05-05 00:53:06,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c691f903fec0b0de9b4 2026-05-05 00:53:06,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:06,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c691f903fec0b0de9b4 HTTP/1.1" 200 None 2026-05-05 00:53:06,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:06,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 00:53:06,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:06,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c691f903fec0b0de9b4 HTTP/1.1" 200 None 2026-05-05 00:53:06,458 - lava_callback - ERROR - Error processing callback for node 69f69c691f903fec0b0de9b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54982 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:53:24,225 - lava_callback - INFO - Received callback for node 69f69c781f903fec0b0de9db INFO: 172.20.0.1:54088 - "POST /node/69f69c781f903fec0b0de9db HTTP/1.0" 202 Accepted 2026-05-05 00:53:24,231 - lava_callback - INFO - Starting async job submit for node 69f69c781f903fec0b0de9db 2026-05-05 00:53:24,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:24,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c781f903fec0b0de9db HTTP/1.1" 200 None 2026-05-05 00:53:24,259 - lava_callback - DEBUG - Processing node 69f69c781f903fec0b0de9db: baseline-arm64-clabbe 2026-05-05 00:53:24,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c781f903fec0b0de9db 2026-05-05 00:53:24,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:24,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c781f903fec0b0de9db HTTP/1.1" 200 None 2026-05-05 00:53:24,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:24,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 00:53:24,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:24,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c781f903fec0b0de9db HTTP/1.1" 200 None 2026-05-05 00:53:24,318 - lava_callback - ERROR - Error processing callback for node 69f69c781f903fec0b0de9db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:53:24,888 - lava_callback - INFO - Received callback for node 69f69c7a1f903fec0b0de9de INFO: 172.20.0.1:54096 - "POST /node/69f69c7a1f903fec0b0de9de HTTP/1.0" 202 Accepted 2026-05-05 00:53:24,895 - lava_callback - INFO - Starting async job submit for node 69f69c7a1f903fec0b0de9de 2026-05-05 00:53:24,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:24,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7a1f903fec0b0de9de HTTP/1.1" 200 None 2026-05-05 00:53:24,913 - lava_callback - DEBUG - Processing node 69f69c7a1f903fec0b0de9de: baseline-arm64-clabbe 2026-05-05 00:53:24,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c7a1f903fec0b0de9de 2026-05-05 00:53:24,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:24,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7a1f903fec0b0de9de HTTP/1.1" 200 None 2026-05-05 00:53:24,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:24,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 00:53:24,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:25,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c7a1f903fec0b0de9de HTTP/1.1" 200 None 2026-05-05 00:53:25,030 - lava_callback - ERROR - Error processing callback for node 69f69c7a1f903fec0b0de9de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:53:25,583 - lava_callback - INFO - Received callback for node 69f69c7b1f903fec0b0de9e1 INFO: 172.20.0.1:54104 - "POST /node/69f69c7b1f903fec0b0de9e1 HTTP/1.0" 202 Accepted 2026-05-05 00:53:25,590 - lava_callback - INFO - Starting async job submit for node 69f69c7b1f903fec0b0de9e1 2026-05-05 00:53:25,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:25,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7b1f903fec0b0de9e1 HTTP/1.1" 200 None 2026-05-05 00:53:25,608 - lava_callback - DEBUG - Processing node 69f69c7b1f903fec0b0de9e1: baseline-arm64-clabbe 2026-05-05 00:53:25,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c7b1f903fec0b0de9e1 2026-05-05 00:53:25,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:25,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7b1f903fec0b0de9e1 HTTP/1.1" 200 None 2026-05-05 00:53:25,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:25,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 00:53:25,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:25,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c7b1f903fec0b0de9e1 HTTP/1.1" 200 None 2026-05-05 00:53:25,659 - lava_callback - ERROR - Error processing callback for node 69f69c7b1f903fec0b0de9e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:53:26,268 - lava_callback - INFO - Received callback for node 69f69c7c1f903fec0b0de9e4 INFO: 172.20.0.1:54110 - "POST /node/69f69c7c1f903fec0b0de9e4 HTTP/1.0" 202 Accepted 2026-05-05 00:53:26,274 - lava_callback - INFO - Starting async job submit for node 69f69c7c1f903fec0b0de9e4 2026-05-05 00:53:26,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:26,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7c1f903fec0b0de9e4 HTTP/1.1" 200 None 2026-05-05 00:53:26,293 - lava_callback - DEBUG - Processing node 69f69c7c1f903fec0b0de9e4: baseline-arm64-clabbe 2026-05-05 00:53:26,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c7c1f903fec0b0de9e4 2026-05-05 00:53:26,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:26,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7c1f903fec0b0de9e4 HTTP/1.1" 200 None 2026-05-05 00:53:26,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:26,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 00:53:26,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:26,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c7c1f903fec0b0de9e4 HTTP/1.1" 200 None 2026-05-05 00:53:26,381 - lava_callback - ERROR - Error processing callback for node 69f69c7c1f903fec0b0de9e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54116 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:53:26,936 - lava_callback - INFO - Received callback for node 69f69c7e1f903fec0b0de9ed INFO: 172.20.0.1:54122 - "POST /node/69f69c7e1f903fec0b0de9ed HTTP/1.0" 202 Accepted 2026-05-05 00:53:26,946 - lava_callback - INFO - Starting async job submit for node 69f69c7e1f903fec0b0de9ed 2026-05-05 00:53:26,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:26,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7e1f903fec0b0de9ed HTTP/1.1" 200 None 2026-05-05 00:53:26,973 - lava_callback - DEBUG - Processing node 69f69c7e1f903fec0b0de9ed: baseline-arm64-clabbe 2026-05-05 00:53:26,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c7e1f903fec0b0de9ed 2026-05-05 00:53:26,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:26,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7e1f903fec0b0de9ed HTTP/1.1" 200 None 2026-05-05 00:53:27,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:27,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 00:53:27,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:27,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c7e1f903fec0b0de9ed HTTP/1.1" 200 None 2026-05-05 00:53:27,066 - lava_callback - ERROR - Error processing callback for node 69f69c7e1f903fec0b0de9ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 00:53:27,611 - lava_callback - INFO - Received callback for node 69f69c7d1f903fec0b0de9ea INFO: 172.20.0.1:54134 - "POST /node/69f69c7d1f903fec0b0de9ea HTTP/1.0" 202 Accepted 2026-05-05 00:53:27,621 - lava_callback - INFO - Starting async job submit for node 69f69c7d1f903fec0b0de9ea 2026-05-05 00:53:27,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:27,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7d1f903fec0b0de9ea HTTP/1.1" 200 None 2026-05-05 00:53:27,647 - lava_callback - DEBUG - Processing node 69f69c7d1f903fec0b0de9ea: baseline-arm64-clabbe 2026-05-05 00:53:27,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69c7d1f903fec0b0de9ea 2026-05-05 00:53:27,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:27,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69c7d1f903fec0b0de9ea HTTP/1.1" 200 None 2026-05-05 00:53:27,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:27,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 00:53:27,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:53:27,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69c7d1f903fec0b0de9ea HTTP/1.1" 200 None 2026-05-05 00:53:27,714 - lava_callback - ERROR - Error processing callback for node 69f69c7d1f903fec0b0de9ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47840 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:54:04,058 - lava_callback - INFO - Received callback for node 69f69ca51f903fec0b0dea9c INFO: 172.20.0.1:40216 - "POST /node/69f69ca51f903fec0b0dea9c HTTP/1.0" 202 Accepted 2026-05-05 00:54:04,064 - lava_callback - INFO - Starting async job submit for node 69f69ca51f903fec0b0dea9c 2026-05-05 00:54:04,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:54:04,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69ca51f903fec0b0dea9c HTTP/1.1" 200 None 2026-05-05 00:54:04,081 - lava_callback - DEBUG - Processing node 69f69ca51f903fec0b0dea9c: baseline-arm64-clabbe 2026-05-05 00:54:04,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69ca51f903fec0b0dea9c 2026-05-05 00:54:04,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:54:04,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69ca51f903fec0b0dea9c HTTP/1.1" 200 None 2026-05-05 00:54:04,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:54:04,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 00:54:04,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:54:04,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69ca51f903fec0b0dea9c HTTP/1.1" 200 None 2026-05-05 00:54:04,134 - lava_callback - ERROR - Error processing callback for node 69f69ca51f903fec0b0dea9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51126 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:55:05,920 - lava_callback - INFO - Received callback for node 69f69cde1f903fec0b0deb36 INFO: 172.20.0.1:46116 - "POST /node/69f69cde1f903fec0b0deb36 HTTP/1.0" 202 Accepted 2026-05-05 00:55:05,927 - lava_callback - INFO - Starting async job submit for node 69f69cde1f903fec0b0deb36 2026-05-05 00:55:05,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:05,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69cde1f903fec0b0deb36 HTTP/1.1" 200 None 2026-05-05 00:55:05,943 - lava_callback - DEBUG - Processing node 69f69cde1f903fec0b0deb36: baseline-arm-baylibre 2026-05-05 00:55:05,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69cde1f903fec0b0deb36 2026-05-05 00:55:05,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:05,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69cde1f903fec0b0deb36 HTTP/1.1" 200 None 2026-05-05 00:55:05,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:05,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 00:55:05,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:06,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69cde1f903fec0b0deb36 HTTP/1.1" 200 None 2026-05-05 00:55:06,006 - lava_callback - ERROR - Error processing callback for node 69f69cde1f903fec0b0deb36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43478 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:55:45,858 - lava_callback - INFO - Received callback for node 69f69d011f903fec0b0deb73 INFO: 172.20.0.1:37046 - "POST /node/69f69d011f903fec0b0deb73 HTTP/1.0" 202 Accepted 2026-05-05 00:55:45,865 - lava_callback - INFO - Starting async job submit for node 69f69d011f903fec0b0deb73 2026-05-05 00:55:45,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:45,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69d011f903fec0b0deb73 HTTP/1.1" 200 None 2026-05-05 00:55:45,883 - lava_callback - DEBUG - Processing node 69f69d011f903fec0b0deb73: baseline-arm-baylibre 2026-05-05 00:55:45,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69d011f903fec0b0deb73 2026-05-05 00:55:45,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:45,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69d011f903fec0b0deb73 HTTP/1.1" 200 None 2026-05-05 00:55:45,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:45,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 00:55:45,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:55:45,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69d011f903fec0b0deb73 HTTP/1.1" 200 None 2026-05-05 00:55:45,973 - lava_callback - ERROR - Error processing callback for node 69f69d011f903fec0b0deb73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:57:43,587 - lava_callback - INFO - Received callback for node 69f94028d09fe59064f05861 INFO: 172.20.0.1:58092 - "POST /node/69f94028d09fe59064f05861 HTTP/1.0" 202 Accepted 2026-05-05 00:57:43,596 - lava_callback - INFO - Starting async job submit for node 69f94028d09fe59064f05861 2026-05-05 00:57:43,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:43,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94028d09fe59064f05861 HTTP/1.1" 200 None 2026-05-05 00:57:43,645 - lava_callback - DEBUG - Processing node 69f94028d09fe59064f05861: baseline-x86 2026-05-05 00:57:43,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:43,992 - lava_callback - INFO - Received callback for node 69f93c69d09fe59064f05514 INFO: 172.20.0.1:58102 - "POST /node/69f93c69d09fe59064f05514 HTTP/1.0" 202 Accepted 2026-05-05 00:57:44,000 - lava_callback - INFO - Starting async job submit for node 69f93c69d09fe59064f05514 2026-05-05 00:57:44,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:44,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c69d09fe59064f05514 HTTP/1.1" 200 None 2026-05-05 00:57:44,124 - lava_callback - DEBUG - Processing node 69f93c69d09fe59064f05514: baseline-arm 2026-05-05 00:57:44,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:44,384 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f94028d09fe59064f05861/log.txt.gz 2026-05-05 00:57:44,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:45,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:45,234 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f94028d09fe59064f05861/lava-logs.yaml 2026-05-05 00:57:45,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:45,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:45,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:45,372 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c69d09fe59064f05514/log.txt.gz 2026-05-05 00:57:45,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:45,429 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f94028d09fe59064f05861/lava_callback.json.gz 2026-05-05 00:57:45,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:45,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94028d09fe59064f05861 HTTP/1.1" 200 None 2026-05-05 00:57:45,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:45,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9372ed09fe59064f02d1d HTTP/1.1" 200 None 2026-05-05 00:57:45,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:45,690 - lava_callback - INFO - Received callback for node 69f9407ad09fe59064f05887 2026-05-05 00:57:45,700 - lava_callback - INFO - Starting async job submit for node 69f9407ad09fe59064f05887 INFO: 172.20.0.1:58110 - "POST /node/69f9407ad09fe59064f05887 HTTP/1.0" 202 Accepted 2026-05-05 00:57:45,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:45,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9407ad09fe59064f05887 HTTP/1.1" 200 None 2026-05-05 00:57:45,770 - lava_callback - DEBUG - Processing node 69f9407ad09fe59064f05887: baseline-arm 2026-05-05 00:57:45,822 - lava_callback - WARNING - Failed to upload log 2026-05-05 00:57:45,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94028d09fe59064f05861 HTTP/1.1" 200 None 2026-05-05 00:57:45,834 - lava_callback - INFO - Completed processing callback for node 69f94028d09fe59064f05861 2026-05-05 00:57:45,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:46,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:46,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:46,067 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f9407ad09fe59064f05887/lava-logs.yaml 2026-05-05 00:57:46,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:46,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:46,150 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f9407ad09fe59064f05887/lava_callback.json.gz 2026-05-05 00:57:46,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:46,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9407ad09fe59064f05887 HTTP/1.1" 200 None 2026-05-05 00:57:46,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:46,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:57:46,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:46,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9407ad09fe59064f05887 HTTP/1.1" 200 None 2026-05-05 00:57:46,220 - lava_callback - INFO - Completed processing callback for node 69f9407ad09fe59064f05887 2026-05-05 00:57:46,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:46,265 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c69d09fe59064f05514/lava-logs.yaml 2026-05-05 00:57:46,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 00:57:46,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 00:57:46,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f93c69d09fe59064f05514/lava_callback.json.gz INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 69f93c69d09fe59064f05514. Transitting job node result: incomplete -> fail 2026-05-05 00:57:46,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:46,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93c69d09fe59064f05514 HTTP/1.1" 200 None 2026-05-05 00:57:46,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:46,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 00:57:46,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:57:46,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93c69d09fe59064f05514 HTTP/1.1" 200 None 2026-05-05 00:57:46,787 - lava_callback - INFO - Completed processing callback for node 69f93c69d09fe59064f05514 INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:58:06,086 - lava_callback - INFO - Received callback for node 69f69d931f903fec0b0debf6 INFO: 172.20.0.1:55388 - "POST /node/69f69d931f903fec0b0debf6 HTTP/1.0" 202 Accepted 2026-05-05 00:58:06,093 - lava_callback - INFO - Starting async job submit for node 69f69d931f903fec0b0debf6 2026-05-05 00:58:06,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:58:06,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69d931f903fec0b0debf6 HTTP/1.1" 200 None 2026-05-05 00:58:06,118 - lava_callback - DEBUG - Processing node 69f69d931f903fec0b0debf6: baseline-arm-baylibre 2026-05-05 00:58:06,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69d931f903fec0b0debf6 2026-05-05 00:58:06,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:58:06,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69d931f903fec0b0debf6 HTTP/1.1" 200 None 2026-05-05 00:58:06,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:58:06,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 00:58:06,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:58:06,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69d931f903fec0b0debf6 HTTP/1.1" 200 None 2026-05-05 00:58:06,215 - lava_callback - ERROR - Error processing callback for node 69f69d931f903fec0b0debf6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55404 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:58:13,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:58:13,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:59:04,388 - lava_callback - INFO - Received callback for node 69f69dc61f903fec0b0dec20 INFO: 172.20.0.1:55388 - "POST /node/69f69dc61f903fec0b0dec20 HTTP/1.0" 202 Accepted 2026-05-05 00:59:04,393 - lava_callback - INFO - Starting async job submit for node 69f69dc61f903fec0b0dec20 2026-05-05 00:59:04,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:04,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69dc61f903fec0b0dec20 HTTP/1.1" 200 None 2026-05-05 00:59:04,419 - lava_callback - DEBUG - Processing node 69f69dc61f903fec0b0dec20: baseline-arm64-clabbe 2026-05-05 00:59:04,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69dc61f903fec0b0dec20 2026-05-05 00:59:04,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69dc61f903fec0b0dec20 HTTP/1.1" 200 None 2026-05-05 00:59:04,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:04,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 00:59:04,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:04,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69dc61f903fec0b0dec20 HTTP/1.1" 200 None 2026-05-05 00:59:04,500 - lava_callback - ERROR - Error processing callback for node 69f69dc61f903fec0b0dec20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK 2026-05-05 00:59:46,607 - lava_callback - INFO - Received callback for node 69f69dff1f903fec0b0dec64 INFO: 172.20.0.1:38798 - "POST /node/69f69dff1f903fec0b0dec64 HTTP/1.0" 202 Accepted 2026-05-05 00:59:46,613 - lava_callback - INFO - Starting async job submit for node 69f69dff1f903fec0b0dec64 2026-05-05 00:59:46,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:46,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69dff1f903fec0b0dec64 HTTP/1.1" 200 None 2026-05-05 00:59:46,629 - lava_callback - DEBUG - Processing node 69f69dff1f903fec0b0dec64: baseline-arm-baylibre 2026-05-05 00:59:46,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69dff1f903fec0b0dec64 2026-05-05 00:59:46,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:46,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69dff1f903fec0b0dec64 HTTP/1.1" 200 None 2026-05-05 00:59:46,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:46,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 00:59:46,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 00:59:46,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69dff1f903fec0b0dec64 HTTP/1.1" 200 None 2026-05-05 00:59:46,693 - lava_callback - ERROR - Error processing callback for node 69f69dff1f903fec0b0dec64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45100 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:01:44,469 - lava_callback - INFO - Received callback for node 69f69e631f903fec0b0dec8d INFO: 172.20.0.1:47940 - "POST /node/69f69e631f903fec0b0dec8d HTTP/1.0" 202 Accepted 2026-05-05 01:01:44,475 - lava_callback - INFO - Starting async job submit for node 69f69e631f903fec0b0dec8d 2026-05-05 01:01:44,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:01:44,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69e631f903fec0b0dec8d HTTP/1.1" 200 None 2026-05-05 01:01:44,490 - lava_callback - DEBUG - Processing node 69f69e631f903fec0b0dec8d: baseline-arm-clabbe 2026-05-05 01:01:44,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69e631f903fec0b0dec8d 2026-05-05 01:01:44,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:01:44,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69e631f903fec0b0dec8d HTTP/1.1" 200 None 2026-05-05 01:01:44,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:01:44,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 01:01:44,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:01:44,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69e631f903fec0b0dec8d HTTP/1.1" 200 None 2026-05-05 01:01:44,536 - lava_callback - ERROR - Error processing callback for node 69f69e631f903fec0b0dec8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:03:04,515 - lava_callback - INFO - Received callback for node 69f69eb91f903fec0b0decbc INFO: 172.20.0.1:60852 - "POST /node/69f69eb91f903fec0b0decbc HTTP/1.0" 202 Accepted 2026-05-05 01:03:04,521 - lava_callback - INFO - Starting async job submit for node 69f69eb91f903fec0b0decbc 2026-05-05 01:03:04,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:03:04,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69eb91f903fec0b0decbc HTTP/1.1" 200 None 2026-05-05 01:03:04,556 - lava_callback - DEBUG - Processing node 69f69eb91f903fec0b0decbc: baseline-arm64-clabbe 2026-05-05 01:03:04,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69eb91f903fec0b0decbc 2026-05-05 01:03:04,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:03:04,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69eb91f903fec0b0decbc HTTP/1.1" 200 None 2026-05-05 01:03:04,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:03:04,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:03:04,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:03:04,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69eb91f903fec0b0decbc HTTP/1.1" 200 None 2026-05-05 01:03:04,668 - lava_callback - ERROR - Error processing callback for node 69f69eb91f903fec0b0decbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43260 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:05:04,641 - lava_callback - INFO - Received callback for node 69f69f3d1f903fec0b0ded19 INFO: 172.20.0.1:40320 - "POST /node/69f69f3d1f903fec0b0ded19 HTTP/1.0" 202 Accepted 2026-05-05 01:05:04,647 - lava_callback - INFO - Starting async job submit for node 69f69f3d1f903fec0b0ded19 2026-05-05 01:05:04,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:04,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69f3d1f903fec0b0ded19 HTTP/1.1" 200 None 2026-05-05 01:05:04,662 - lava_callback - DEBUG - Processing node 69f69f3d1f903fec0b0ded19: baseline-arm-clabbe 2026-05-05 01:05:04,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69f3d1f903fec0b0ded19 2026-05-05 01:05:04,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:04,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69f3d1f903fec0b0ded19 HTTP/1.1" 200 None 2026-05-05 01:05:04,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:04,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 01:05:04,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:04,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69f3d1f903fec0b0ded19 HTTP/1.1" 200 None 2026-05-05 01:05:04,730 - lava_callback - ERROR - Error processing callback for node 69f69f3d1f903fec0b0ded19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54288 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:05:46,340 - lava_callback - INFO - Received callback for node 69f9425ed09fe59064f0598a INFO: 172.20.0.1:45006 - "POST /node/69f9425ed09fe59064f0598a HTTP/1.0" 202 Accepted 2026-05-05 01:05:46,352 - lava_callback - INFO - Starting async job submit for node 69f9425ed09fe59064f0598a 2026-05-05 01:05:46,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:46,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9425ed09fe59064f0598a HTTP/1.1" 200 None 2026-05-05 01:05:46,397 - lava_callback - DEBUG - Processing node 69f9425ed09fe59064f0598a: baseline-arm64 2026-05-05 01:05:46,405 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:05:46,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45010 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:05:46,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:05:46,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425ed09fe59064f0598a/lava-logs.yaml 2026-05-05 01:05:46,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:05:46,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:05:46,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425ed09fe59064f0598a/lava_callback.json.gz 2026-05-05 01:05:46,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:46,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9425ed09fe59064f0598a HTTP/1.1" 200 None 2026-05-05 01:05:46,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:46,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:05:46,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:05:46,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9425ed09fe59064f0598a HTTP/1.1" 200 None 2026-05-05 01:05:46,688 - lava_callback - INFO - Completed processing callback for node 69f9425ed09fe59064f0598a INFO: 172.20.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42978 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:06:13,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:13,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57092 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:06:19,252 - lava_callback - INFO - Received callback for node 69f94269d09fe59064f059ab INFO: 172.20.0.1:57098 - "POST /node/69f94269d09fe59064f059ab HTTP/1.0" 202 Accepted 2026-05-05 01:06:19,263 - lava_callback - INFO - Starting async job submit for node 69f94269d09fe59064f059ab 2026-05-05 01:06:19,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:19,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94269d09fe59064f059ab HTTP/1.1" 200 None 2026-05-05 01:06:19,309 - lava_callback - DEBUG - Processing node 69f94269d09fe59064f059ab: baseline-arm64 2026-05-05 01:06:19,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:19,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:19,603 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94269d09fe59064f059ab/log.txt.gz 2026-05-05 01:06:19,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:19,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:19,861 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94269d09fe59064f059ab/lava-logs.yaml 2026-05-05 01:06:19,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:19,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:19,999 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94269d09fe59064f059ab/lava_callback.json.gz 2026-05-05 01:06:20,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:20,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94269d09fe59064f059ab HTTP/1.1" 200 None 2026-05-05 01:06:20,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:20,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:06:20,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:20,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94269d09fe59064f059ab HTTP/1.1" 200 None 2026-05-05 01:06:20,078 - lava_callback - INFO - Completed processing callback for node 69f94269d09fe59064f059ab 2026-05-05 01:06:24,650 - lava_callback - INFO - Received callback for node 69f94277d09fe59064f059fb INFO: 172.20.0.1:35708 - "POST /node/69f94277d09fe59064f059fb HTTP/1.0" 202 Accepted 2026-05-05 01:06:24,657 - lava_callback - INFO - Starting async job submit for node 69f94277d09fe59064f059fb 2026-05-05 01:06:24,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:24,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94277d09fe59064f059fb HTTP/1.1" 200 None 2026-05-05 01:06:24,675 - lava_callback - DEBUG - Processing node 69f94277d09fe59064f059fb: baseline-nfs-arm64 2026-05-05 01:06:24,676 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:06:24,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:24,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:24,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94277d09fe59064f059fb/lava-logs.yaml 2026-05-05 01:06:24,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:24,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:24,908 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94277d09fe59064f059fb/lava_callback.json.gz 2026-05-05 01:06:24,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:24,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94277d09fe59064f059fb HTTP/1.1" 200 None 2026-05-05 01:06:24,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:24,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:06:24,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:24,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94277d09fe59064f059fb HTTP/1.1" 200 None 2026-05-05 01:06:24,973 - lava_callback - INFO - Completed processing callback for node 69f94277d09fe59064f059fb INFO: 172.20.0.1:35716 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:06:26,599 - lava_callback - INFO - Received callback for node 69f69f8d1f903fec0b0ded3a INFO: 172.20.0.1:35720 - "POST /node/69f69f8d1f903fec0b0ded3a HTTP/1.0" 202 Accepted 2026-05-05 01:06:26,609 - lava_callback - INFO - Starting async job submit for node 69f69f8d1f903fec0b0ded3a 2026-05-05 01:06:26,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:26,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69f8d1f903fec0b0ded3a HTTP/1.1" 200 None 2026-05-05 01:06:26,634 - lava_callback - DEBUG - Processing node 69f69f8d1f903fec0b0ded3a: baseline-arm64-baylibre 2026-05-05 01:06:26,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69f8d1f903fec0b0ded3a 2026-05-05 01:06:26,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:26,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69f8d1f903fec0b0ded3a HTTP/1.1" 200 None 2026-05-05 01:06:26,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:26,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:06:26,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:26,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69f8d1f903fec0b0ded3a HTTP/1.1" 200 None 2026-05-05 01:06:26,708 - lava_callback - ERROR - Error processing callback for node 69f69f8d1f903fec0b0ded3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:06:40,500 - lava_callback - INFO - Received callback for node 69f94261d09fe59064f05996 INFO: 172.20.0.1:49106 - "POST /node/69f94261d09fe59064f05996 HTTP/1.0" 202 Accepted 2026-05-05 01:06:40,507 - lava_callback - INFO - Starting async job submit for node 69f94261d09fe59064f05996 2026-05-05 01:06:40,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:40,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94261d09fe59064f05996 HTTP/1.1" 200 None 2026-05-05 01:06:40,570 - lava_callback - DEBUG - Processing node 69f94261d09fe59064f05996: baseline-arm64 2026-05-05 01:06:40,704 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:06:40,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:40,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:40,900 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94261d09fe59064f05996/lava-logs.yaml 2026-05-05 01:06:40,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:41,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:41,031 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94261d09fe59064f05996/lava_callback.json.gz 2026-05-05 01:06:41,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:41,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94261d09fe59064f05996 HTTP/1.1" 200 None 2026-05-05 01:06:41,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:41,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:06:41,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:41,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94261d09fe59064f05996 HTTP/1.1" 200 None 2026-05-05 01:06:41,210 - lava_callback - INFO - Completed processing callback for node 69f94261d09fe59064f05996 2026-05-05 01:06:42,624 - lava_callback - INFO - Received callback for node 69f9425fd09fe59064f05990 INFO: 172.20.0.1:49108 - "POST /node/69f9425fd09fe59064f05990 HTTP/1.0" 202 Accepted 2026-05-05 01:06:42,631 - lava_callback - INFO - Starting async job submit for node 69f9425fd09fe59064f05990 2026-05-05 01:06:42,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:42,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9425fd09fe59064f05990 HTTP/1.1" 200 None 2026-05-05 01:06:42,668 - lava_callback - DEBUG - Processing node 69f9425fd09fe59064f05990: baseline-arm64 2026-05-05 01:06:43,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:43,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:43,128 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425fd09fe59064f05990/log.txt.gz 2026-05-05 01:06:43,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:43,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:43,532 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425fd09fe59064f05990/lava-logs.yaml 2026-05-05 01:06:43,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:43,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:43,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425fd09fe59064f05990/lava_callback.json.gz 2026-05-05 01:06:43,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:43,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:43,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9425fd09fe59064f05990 HTTP/1.1" 200 None 2026-05-05 01:06:43,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:43,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 01:06:43,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:06:43,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:43,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9425fd09fe59064f05990 HTTP/1.1" 200 None 2026-05-05 01:06:43,749 - lava_callback - INFO - Completed processing callback for node 69f9425fd09fe59064f05990 INFO: 172.20.0.1:49110 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:06:52,183 - lava_callback - INFO - Received callback for node 69f94265d09fe59064f0599f INFO: 172.20.0.1:56358 - "POST /node/69f94265d09fe59064f0599f HTTP/1.0" 202 Accepted 2026-05-05 01:06:52,189 - lava_callback - INFO - Starting async job submit for node 69f94265d09fe59064f0599f 2026-05-05 01:06:52,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:52,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94265d09fe59064f0599f HTTP/1.1" 200 None 2026-05-05 01:06:52,225 - lava_callback - DEBUG - Processing node 69f94265d09fe59064f0599f: baseline-arm64 2026-05-05 01:06:52,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:52,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:52,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94265d09fe59064f0599f/log.txt.gz 2026-05-05 01:06:52,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:52,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:52,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94265d09fe59064f0599f/lava-logs.yaml 2026-05-05 01:06:52,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:06:53,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:06:53,052 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94265d09fe59064f0599f/lava_callback.json.gz 2026-05-05 01:06:53,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:53,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94265d09fe59064f0599f HTTP/1.1" 200 None 2026-05-05 01:06:53,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:53,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:06:53,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:06:53,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94265d09fe59064f0599f HTTP/1.1" 200 None 2026-05-05 01:06:53,169 - lava_callback - INFO - Completed processing callback for node 69f94265d09fe59064f0599f INFO: 172.20.0.1:56374 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:07:02,311 - lava_callback - INFO - Received callback for node 69f9426ed09fe59064f059df INFO: 172.20.0.1:45084 - "POST /node/69f9426ed09fe59064f059df HTTP/1.0" 202 Accepted 2026-05-05 01:07:02,318 - lava_callback - INFO - Starting async job submit for node 69f9426ed09fe59064f059df 2026-05-05 01:07:02,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:02,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426ed09fe59064f059df HTTP/1.1" 200 None 2026-05-05 01:07:02,383 - lava_callback - DEBUG - Processing node 69f9426ed09fe59064f059df: baseline-arm64 2026-05-05 01:07:02,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:02,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:02,767 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426ed09fe59064f059df/log.txt.gz 2026-05-05 01:07:02,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:03,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:03,046 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426ed09fe59064f059df/lava-logs.yaml 2026-05-05 01:07:03,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:03,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:03,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426ed09fe59064f059df/lava_callback.json.gz 2026-05-05 01:07:03,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:03,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426ed09fe59064f059df HTTP/1.1" 200 None 2026-05-05 01:07:03,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:03,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:03,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:03,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9426ed09fe59064f059df HTTP/1.1" 200 None 2026-05-05 01:07:03,233 - lava_callback - INFO - Completed processing callback for node 69f9426ed09fe59064f059df 2026-05-05 01:07:04,508 - lava_callback - INFO - Received callback for node 69f94266d09fe59064f059a2 INFO: 172.20.0.1:45090 - "POST /node/69f94266d09fe59064f059a2 HTTP/1.0" 202 Accepted 2026-05-05 01:07:04,514 - lava_callback - INFO - Starting async job submit for node 69f94266d09fe59064f059a2 2026-05-05 01:07:04,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:04,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94266d09fe59064f059a2 HTTP/1.1" 200 None 2026-05-05 01:07:04,550 - lava_callback - DEBUG - Processing node 69f94266d09fe59064f059a2: baseline-arm64 2026-05-05 01:07:04,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:05,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:05,078 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94266d09fe59064f059a2/log.txt.gz 2026-05-05 01:07:05,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:05,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:05,478 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94266d09fe59064f059a2/lava-logs.yaml 2026-05-05 01:07:05,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:05,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:05,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94266d09fe59064f059a2/lava_callback.json.gz 2026-05-05 01:07:05,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:05,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94266d09fe59064f059a2 HTTP/1.1" 200 None 2026-05-05 01:07:05,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:05,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:05,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:05,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94266d09fe59064f059a2 HTTP/1.1" 200 None 2026-05-05 01:07:05,700 - lava_callback - INFO - Completed processing callback for node 69f94266d09fe59064f059a2 2026-05-05 01:07:06,053 - lava_callback - INFO - Received callback for node 69f9426ad09fe59064f059ae INFO: 172.20.0.1:45106 - "POST /node/69f9426ad09fe59064f059ae HTTP/1.0" 202 Accepted 2026-05-05 01:07:06,059 - lava_callback - INFO - Starting async job submit for node 69f9426ad09fe59064f059ae 2026-05-05 01:07:06,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:06,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426ad09fe59064f059ae HTTP/1.1" 200 None 2026-05-05 01:07:06,096 - lava_callback - DEBUG - Processing node 69f9426ad09fe59064f059ae: baseline-arm64 2026-05-05 01:07:06,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:06,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:06,422 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426ad09fe59064f059ae/log.txt.gz INFO: 172.20.0.1:45114 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:07:06,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:06,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:06,693 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426ad09fe59064f059ae/lava-logs.yaml 2026-05-05 01:07:06,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:06,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:06,861 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426ad09fe59064f059ae/lava_callback.json.gz 2026-05-05 01:07:06,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:06,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426ad09fe59064f059ae HTTP/1.1" 200 None 2026-05-05 01:07:06,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:06,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:06,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:06,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9426ad09fe59064f059ae HTTP/1.1" 200 None 2026-05-05 01:07:06,935 - lava_callback - INFO - Completed processing callback for node 69f9426ad09fe59064f059ae 2026-05-05 01:07:07,461 - lava_callback - INFO - Received callback for node 69f9426fd09fe59064f059e2 INFO: 172.20.0.1:45130 - "POST /node/69f9426fd09fe59064f059e2 HTTP/1.0" 202 Accepted 2026-05-05 01:07:07,467 - lava_callback - INFO - Starting async job submit for node 69f9426fd09fe59064f059e2 2026-05-05 01:07:07,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:07,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426fd09fe59064f059e2 HTTP/1.1" 200 None 2026-05-05 01:07:07,505 - lava_callback - DEBUG - Processing node 69f9426fd09fe59064f059e2: baseline-arm64 2026-05-05 01:07:07,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:07,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:07,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426fd09fe59064f059e2/log.txt.gz 2026-05-05 01:07:08,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:08,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:08,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426fd09fe59064f059e2/lava-logs.yaml 2026-05-05 01:07:08,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:08,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:08,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426fd09fe59064f059e2/lava_callback.json.gz 2026-05-05 01:07:08,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:08,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426fd09fe59064f059e2 HTTP/1.1" 200 None 2026-05-05 01:07:08,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:08,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:08,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:08,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9426fd09fe59064f059e2 HTTP/1.1" 200 None 2026-05-05 01:07:08,552 - lava_callback - INFO - Completed processing callback for node 69f9426fd09fe59064f059e2 2026-05-05 01:07:09,880 - lava_callback - INFO - Received callback for node 69f9426dd09fe59064f059dc INFO: 172.20.0.1:42192 - "POST /node/69f9426dd09fe59064f059dc HTTP/1.0" 202 Accepted 2026-05-05 01:07:09,887 - lava_callback - INFO - Starting async job submit for node 69f9426dd09fe59064f059dc 2026-05-05 01:07:09,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:09,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426dd09fe59064f059dc HTTP/1.1" 200 None 2026-05-05 01:07:09,923 - lava_callback - DEBUG - Processing node 69f9426dd09fe59064f059dc: baseline-arm64 2026-05-05 01:07:10,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:10,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:10,449 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426dd09fe59064f059dc/log.txt.gz 2026-05-05 01:07:10,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:10,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:10,734 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426dd09fe59064f059dc/lava-logs.yaml 2026-05-05 01:07:10,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:10,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:10,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9426dd09fe59064f059dc/lava_callback.json.gz 2026-05-05 01:07:10,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:10,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9426dd09fe59064f059dc HTTP/1.1" 200 None 2026-05-05 01:07:10,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:10,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:10,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:10,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9426dd09fe59064f059dc HTTP/1.1" 200 None 2026-05-05 01:07:10,933 - lava_callback - INFO - Completed processing callback for node 69f9426dd09fe59064f059dc 2026-05-05 01:07:13,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:13,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:42200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42206 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:07:17,652 - lava_callback - INFO - Received callback for node 69f94268d09fe59064f059a8 INFO: 172.20.0.1:42208 - "POST /node/69f94268d09fe59064f059a8 HTTP/1.0" 202 Accepted 2026-05-05 01:07:17,659 - lava_callback - INFO - Starting async job submit for node 69f94268d09fe59064f059a8 2026-05-05 01:07:17,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:17,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94268d09fe59064f059a8 HTTP/1.1" 200 None 2026-05-05 01:07:17,696 - lava_callback - DEBUG - Processing node 69f94268d09fe59064f059a8: baseline-arm64 2026-05-05 01:07:18,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:18,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:18,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94268d09fe59064f059a8/log.txt.gz 2026-05-05 01:07:18,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:18,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:18,790 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94268d09fe59064f059a8/lava-logs.yaml 2026-05-05 01:07:18,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:18,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:18,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94268d09fe59064f059a8/lava_callback.json.gz 2026-05-05 01:07:18,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:18,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94268d09fe59064f059a8 HTTP/1.1" 200 None 2026-05-05 01:07:18,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:18,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:18,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:18,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94268d09fe59064f059a8 HTTP/1.1" 200 None 2026-05-05 01:07:19,004 - lava_callback - INFO - Completed processing callback for node 69f94268d09fe59064f059a8 2026-05-05 01:07:20,180 - lava_callback - INFO - Received callback for node 69f9427cd09fe59064f05a07 INFO: 172.20.0.1:39748 - "POST /node/69f9427cd09fe59064f05a07 HTTP/1.0" 202 Accepted 2026-05-05 01:07:20,187 - lava_callback - INFO - Starting async job submit for node 69f9427cd09fe59064f05a07 2026-05-05 01:07:20,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:20,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9427cd09fe59064f05a07 HTTP/1.1" 200 None 2026-05-05 01:07:20,225 - lava_callback - DEBUG - Processing node 69f9427cd09fe59064f05a07: baseline-nfs-arm64 2026-05-05 01:07:20,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:20,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:20,658 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427cd09fe59064f05a07/log.txt.gz 2026-05-05 01:07:20,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:21,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:21,029 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427cd09fe59064f05a07/lava-logs.yaml 2026-05-05 01:07:21,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:21,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:21,127 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427cd09fe59064f05a07/lava_callback.json.gz 2026-05-05 01:07:21,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:21,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9427cd09fe59064f05a07 HTTP/1.1" 200 None 2026-05-05 01:07:21,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:21,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:21,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:21,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9427cd09fe59064f05a07 HTTP/1.1" 200 None 2026-05-05 01:07:21,207 - lava_callback - INFO - Completed processing callback for node 69f9427cd09fe59064f05a07 INFO: 172.20.0.1:39758 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:07:26,607 - lava_callback - INFO - Received callback for node 69f9425ed09fe59064f0598d INFO: 172.20.0.1:39762 - "POST /node/69f9425ed09fe59064f0598d HTTP/1.0" 202 Accepted 2026-05-05 01:07:26,613 - lava_callback - INFO - Starting async job submit for node 69f9425ed09fe59064f0598d 2026-05-05 01:07:26,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:26,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9425ed09fe59064f0598d HTTP/1.1" 200 None 2026-05-05 01:07:26,648 - lava_callback - DEBUG - Processing node 69f9425ed09fe59064f0598d: baseline-arm64 2026-05-05 01:07:26,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:26,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:26,992 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425ed09fe59064f0598d/log.txt.gz 2026-05-05 01:07:27,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:27,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:27,261 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425ed09fe59064f0598d/lava-logs.yaml 2026-05-05 01:07:27,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:27,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:27,376 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f9425ed09fe59064f0598d/lava_callback.json.gz 2026-05-05 01:07:27,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:27,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9425ed09fe59064f0598d HTTP/1.1" 200 None 2026-05-05 01:07:27,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:27,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:27,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:27,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9425ed09fe59064f0598d HTTP/1.1" 200 None 2026-05-05 01:07:27,450 - lava_callback - INFO - Completed processing callback for node 69f9425ed09fe59064f0598d 2026-05-05 01:07:28,165 - lava_callback - INFO - Received callback for node 69f94260d09fe59064f05993 INFO: 172.20.0.1:39766 - "POST /node/69f94260d09fe59064f05993 HTTP/1.0" 202 Accepted 2026-05-05 01:07:28,172 - lava_callback - INFO - Starting async job submit for node 69f94260d09fe59064f05993 2026-05-05 01:07:28,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:28,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94260d09fe59064f05993 HTTP/1.1" 200 None 2026-05-05 01:07:28,208 - lava_callback - DEBUG - Processing node 69f94260d09fe59064f05993: baseline-arm64 2026-05-05 01:07:28,264 - lava_callback - INFO - Received callback for node 69f94278d09fe59064f059fe INFO: 172.20.0.1:39772 - "POST /node/69f94278d09fe59064f059fe HTTP/1.0" 202 Accepted 2026-05-05 01:07:28,291 - lava_callback - INFO - Starting async job submit for node 69f94278d09fe59064f059fe 2026-05-05 01:07:28,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:28,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94278d09fe59064f059fe HTTP/1.1" 200 None 2026-05-05 01:07:28,471 - lava_callback - DEBUG - Processing node 69f94278d09fe59064f059fe: baseline-nfs-arm64 INFO: 172.20.0.1:39786 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:07:28,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:28,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:28,807 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94260d09fe59064f05993/log.txt.gz 2026-05-05 01:07:28,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:28,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:29,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:29,056 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94278d09fe59064f059fe/log.txt.gz 2026-05-05 01:07:29,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:29,175 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94260d09fe59064f05993/lava-logs.yaml 2026-05-05 01:07:29,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:29,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:29,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:29,353 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94260d09fe59064f05993/lava_callback.json.gz 2026-05-05 01:07:29,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:29,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94260d09fe59064f05993 HTTP/1.1" 200 None 2026-05-05 01:07:29,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:29,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:29,411 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94278d09fe59064f059fe/lava-logs.yaml 2026-05-05 01:07:29,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:29,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:29,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:29,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94260d09fe59064f05993 HTTP/1.1" 200 None 2026-05-05 01:07:29,441 - lava_callback - INFO - Completed processing callback for node 69f94260d09fe59064f05993 2026-05-05 01:07:29,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:29,544 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94278d09fe59064f059fe/lava_callback.json.gz 2026-05-05 01:07:29,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:29,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94278d09fe59064f059fe HTTP/1.1" 200 None 2026-05-05 01:07:29,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:29,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:29,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:29,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94278d09fe59064f059fe HTTP/1.1" 200 None 2026-05-05 01:07:29,640 - lava_callback - INFO - Completed processing callback for node 69f94278d09fe59064f059fe 2026-05-05 01:07:30,342 - lava_callback - INFO - Received callback for node 69f94275d09fe59064f059f4 INFO: 172.20.0.1:46922 - "POST /node/69f94275d09fe59064f059f4 HTTP/1.0" 202 Accepted 2026-05-05 01:07:30,352 - lava_callback - INFO - Starting async job submit for node 69f94275d09fe59064f059f4 2026-05-05 01:07:30,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:30,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94275d09fe59064f059f4 HTTP/1.1" 200 None 2026-05-05 01:07:30,420 - lava_callback - DEBUG - Processing node 69f94275d09fe59064f059f4: baseline-nfs-arm64 2026-05-05 01:07:30,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:31,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:31,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94275d09fe59064f059f4/log.txt.gz 2026-05-05 01:07:31,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:31,405 - lava_callback - INFO - Received callback for node 69f94267d09fe59064f059a5 INFO: 172.20.0.1:46934 - "POST /node/69f94267d09fe59064f059a5 HTTP/1.0" 202 Accepted 2026-05-05 01:07:31,412 - lava_callback - INFO - Starting async job submit for node 69f94267d09fe59064f059a5 2026-05-05 01:07:31,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:31,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94267d09fe59064f059a5 HTTP/1.1" 200 None 2026-05-05 01:07:31,484 - lava_callback - DEBUG - Processing node 69f94267d09fe59064f059a5: baseline-arm64 2026-05-05 01:07:31,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:31,567 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94275d09fe59064f059f4/lava-logs.yaml 2026-05-05 01:07:31,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:31,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:31,900 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94275d09fe59064f059f4/lava_callback.json.gz 2026-05-05 01:07:31,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:32,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:32,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94275d09fe59064f059f4 HTTP/1.1" 200 None 2026-05-05 01:07:32,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:32,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:32,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:32,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94275d09fe59064f059f4 HTTP/1.1" 200 None 2026-05-05 01:07:32,064 - lava_callback - INFO - Completed processing callback for node 69f94275d09fe59064f059f4 2026-05-05 01:07:32,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:32,137 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94267d09fe59064f059a5/log.txt.gz 2026-05-05 01:07:32,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:32,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:32,568 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94267d09fe59064f059a5/lava-logs.yaml 2026-05-05 01:07:32,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:32,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:32,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94267d09fe59064f059a5/lava_callback.json.gz 2026-05-05 01:07:32,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:32,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94267d09fe59064f059a5 HTTP/1.1" 200 None 2026-05-05 01:07:32,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:32,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:32,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:32,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94267d09fe59064f059a5 HTTP/1.1" 200 None 2026-05-05 01:07:32,760 - lava_callback - INFO - Completed processing callback for node 69f94267d09fe59064f059a5 2026-05-05 01:07:34,311 - lava_callback - INFO - Received callback for node 69f942a2d09fe59064f05a8c INFO: 172.20.0.1:46950 - "POST /node/69f942a2d09fe59064f05a8c HTTP/1.0" 202 Accepted 2026-05-05 01:07:34,317 - lava_callback - INFO - Starting async job submit for node 69f942a2d09fe59064f05a8c 2026-05-05 01:07:34,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:34,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a2d09fe59064f05a8c HTTP/1.1" 200 None 2026-05-05 01:07:34,358 - lava_callback - DEBUG - Processing node 69f942a2d09fe59064f05a8c: baseline-arm64-foundriesio 2026-05-05 01:07:34,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:34,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:34,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f942a2d09fe59064f05a8c/log.txt.gz 2026-05-05 01:07:34,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:34,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:34,998 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f942a2d09fe59064f05a8c/lava-logs.yaml 2026-05-05 01:07:35,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:35,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:35,104 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f942a2d09fe59064f05a8c/lava_callback.json.gz 2026-05-05 01:07:35,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:35,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a2d09fe59064f05a8c HTTP/1.1" 200 None 2026-05-05 01:07:35,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:35,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:35,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:35,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f942a2d09fe59064f05a8c HTTP/1.1" 200 None 2026-05-05 01:07:35,182 - lava_callback - INFO - Completed processing callback for node 69f942a2d09fe59064f05a8c INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:07:37,779 - lava_callback - INFO - Received callback for node 69f9427bd09fe59064f05a04 INFO: 172.20.0.1:46970 - "POST /node/69f9427bd09fe59064f05a04 HTTP/1.0" 202 Accepted 2026-05-05 01:07:37,786 - lava_callback - INFO - Starting async job submit for node 69f9427bd09fe59064f05a04 2026-05-05 01:07:37,810 - lava_callback - INFO - Received callback for node 69f94276d09fe59064f059f8 2026-05-05 01:07:37,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:46972 - "POST /node/69f94276d09fe59064f059f8 HTTP/1.0" 202 Accepted 2026-05-05 01:07:37,818 - lava_callback - INFO - Starting async job submit for node 69f94276d09fe59064f059f8 2026-05-05 01:07:37,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:37,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9427bd09fe59064f05a04 HTTP/1.1" 200 None 2026-05-05 01:07:37,852 - lava_callback - DEBUG - Processing node 69f9427bd09fe59064f05a04: baseline-nfs-arm64 2026-05-05 01:07:37,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94276d09fe59064f059f8 HTTP/1.1" 200 None 2026-05-05 01:07:37,896 - lava_callback - DEBUG - Processing node 69f94276d09fe59064f059f8: baseline-nfs-arm64 2026-05-05 01:07:38,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:38,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:38,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427bd09fe59064f05a04/log.txt.gz 2026-05-05 01:07:39,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:39,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:39,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:39,351 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94276d09fe59064f059f8/log.txt.gz 2026-05-05 01:07:39,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:39,495 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427bd09fe59064f05a04/lava-logs.yaml 2026-05-05 01:07:39,538 - lava_callback - INFO - Received callback for node 69f94271d09fe59064f059e8 INFO: 172.20.0.1:46980 - "POST /node/69f94271d09fe59064f059e8 HTTP/1.0" 202 Accepted 2026-05-05 01:07:39,565 - lava_callback - INFO - Starting async job submit for node 69f94271d09fe59064f059e8 2026-05-05 01:07:39,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:39,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:39,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94271d09fe59064f059e8 HTTP/1.1" 200 None 2026-05-05 01:07:39,706 - lava_callback - DEBUG - Processing node 69f94271d09fe59064f059e8: baseline-nfs-arm64 2026-05-05 01:07:39,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:39,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:39,971 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427bd09fe59064f05a04/lava_callback.json.gz 2026-05-05 01:07:40,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:40,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9427bd09fe59064f05a04 HTTP/1.1" 200 None 2026-05-05 01:07:40,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:40,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:40,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:40,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:40,319 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94276d09fe59064f059f8/lava-logs.yaml 2026-05-05 01:07:40,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:40,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:40,354 - lava_callback - INFO - Received callback for node 69f94262d09fe59064f05999 2026-05-05 01:07:40,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9427bd09fe59064f05a04 HTTP/1.1" 200 None 2026-05-05 01:07:40,380 - lava_callback - INFO - Completed processing callback for node 69f9427bd09fe59064f05a04 INFO: 172.20.0.1:37088 - "POST /node/69f94262d09fe59064f05999 HTTP/1.0" 202 Accepted 2026-05-05 01:07:40,387 - lava_callback - INFO - Starting async job submit for node 69f94262d09fe59064f05999 2026-05-05 01:07:40,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:40,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94262d09fe59064f05999 HTTP/1.1" 200 None 2026-05-05 01:07:40,482 - lava_callback - DEBUG - Processing node 69f94262d09fe59064f05999: baseline-arm64 2026-05-05 01:07:40,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:40,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:40,578 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94276d09fe59064f059f8/lava_callback.json.gz 2026-05-05 01:07:40,644 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94271d09fe59064f059e8/log.txt.gz 2026-05-05 01:07:40,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:40,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94276d09fe59064f059f8 HTTP/1.1" 200 None 2026-05-05 01:07:40,973 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:07:41,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:41,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:41,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:41,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94276d09fe59064f059f8 HTTP/1.1" 200 None 2026-05-05 01:07:41,176 - lava_callback - INFO - Completed processing callback for node 69f94276d09fe59064f059f8 2026-05-05 01:07:41,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:41,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94271d09fe59064f059e8/lava-logs.yaml 2026-05-05 01:07:41,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:41,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:41,361 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94262d09fe59064f05999/lava-logs.yaml 2026-05-05 01:07:41,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:41,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:41,431 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94271d09fe59064f059e8/lava_callback.json.gz 2026-05-05 01:07:41,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:41,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94262d09fe59064f05999/lava_callback.json.gz 2026-05-05 01:07:41,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94271d09fe59064f059e8 HTTP/1.1" 200 None 2026-05-05 01:07:41,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:41,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94262d09fe59064f05999 HTTP/1.1" 200 None 2026-05-05 01:07:41,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:41,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:41,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94271d09fe59064f059e8 HTTP/1.1" 200 None 2026-05-05 01:07:41,553 - lava_callback - INFO - Completed processing callback for node 69f94271d09fe59064f059e8 2026-05-05 01:07:41,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94262d09fe59064f05999 HTTP/1.1" 200 None 2026-05-05 01:07:41,691 - lava_callback - INFO - Completed processing callback for node 69f94262d09fe59064f05999 2026-05-05 01:07:43,062 - lava_callback - INFO - Received callback for node 69f94273d09fe59064f059ee INFO: 172.20.0.1:37100 - "POST /node/69f94273d09fe59064f059ee HTTP/1.0" 202 Accepted 2026-05-05 01:07:43,068 - lava_callback - INFO - Starting async job submit for node 69f94273d09fe59064f059ee 2026-05-05 01:07:43,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:43,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94273d09fe59064f059ee HTTP/1.1" 200 None 2026-05-05 01:07:43,114 - lava_callback - DEBUG - Processing node 69f94273d09fe59064f059ee: baseline-nfs-arm64 2026-05-05 01:07:43,340 - lava_callback - INFO - Received callback for node 69f9429dd09fe59064f05a7e INFO: 172.20.0.1:37116 - "POST /node/69f9429dd09fe59064f05a7e HTTP/1.0" 202 Accepted 2026-05-05 01:07:43,356 - lava_callback - INFO - Starting async job submit for node 69f9429dd09fe59064f05a7e 2026-05-05 01:07:43,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:43,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:43,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9429dd09fe59064f05a7e HTTP/1.1" 200 None 2026-05-05 01:07:43,511 - lava_callback - DEBUG - Processing node 69f9429dd09fe59064f05a7e: baseline-arm64-kontron 2026-05-05 01:07:43,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:43,733 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94273d09fe59064f059ee/log.txt.gz 2026-05-05 01:07:43,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:43,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:43,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-05-05 01:07:44,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:44,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:44,080 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69f9429dd09fe59064f05a7e/log.txt.gz 2026-05-05 01:07:44,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:44,215 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94273d09fe59064f059ee/lava-logs.yaml 2026-05-05 01:07:44,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:44,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:44,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:44,363 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94273d09fe59064f059ee/lava_callback.json.gz 2026-05-05 01:07:44,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:44,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94273d09fe59064f059ee HTTP/1.1" 200 None 2026-05-05 01:07:44,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:44,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:44,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:44,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:44,419 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69f9429dd09fe59064f05a7e/lava-logs.yaml 2026-05-05 01:07:44,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:44,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94273d09fe59064f059ee HTTP/1.1" 200 None 2026-05-05 01:07:44,442 - lava_callback - INFO - Completed processing callback for node 69f94273d09fe59064f059ee 2026-05-05 01:07:44,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:44,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69f9429dd09fe59064f05a7e/lava_callback.json.gz 2026-05-05 01:07:44,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:44,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9429dd09fe59064f05a7e HTTP/1.1" 200 None 2026-05-05 01:07:44,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:44,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:44,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:44,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9429dd09fe59064f05a7e HTTP/1.1" 200 None 2026-05-05 01:07:44,700 - lava_callback - INFO - Completed processing callback for node 69f9429dd09fe59064f05a7e INFO: 172.20.0.1:37124 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:07:46,835 - lava_callback - INFO - Received callback for node 69f9427dd09fe59064f05a0a INFO: 172.20.0.1:37138 - "POST /node/69f9427dd09fe59064f05a0a HTTP/1.0" 202 Accepted 2026-05-05 01:07:46,842 - lava_callback - INFO - Starting async job submit for node 69f9427dd09fe59064f05a0a 2026-05-05 01:07:46,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9427dd09fe59064f05a0a HTTP/1.1" 200 None 2026-05-05 01:07:46,879 - lava_callback - DEBUG - Processing node 69f9427dd09fe59064f05a0a: baseline-nfs-arm64 2026-05-05 01:07:47,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:47,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:47,441 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427dd09fe59064f05a0a/log.txt.gz 2026-05-05 01:07:47,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:47,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:47,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427dd09fe59064f05a0a/lava-logs.yaml 2026-05-05 01:07:47,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:47,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:47,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9427dd09fe59064f05a0a/lava_callback.json.gz 2026-05-05 01:07:47,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:47,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9427dd09fe59064f05a0a HTTP/1.1" 200 None 2026-05-05 01:07:47,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:48,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:48,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:48,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9427dd09fe59064f05a0a HTTP/1.1" 200 None 2026-05-05 01:07:48,024 - lava_callback - INFO - Completed processing callback for node 69f9427dd09fe59064f05a0a 2026-05-05 01:07:53,259 - lava_callback - INFO - Received callback for node 69f9429ed09fe59064f05a81 INFO: 172.20.0.1:50990 - "POST /node/69f9429ed09fe59064f05a81 HTTP/1.0" 202 Accepted 2026-05-05 01:07:53,265 - lava_callback - INFO - Starting async job submit for node 69f9429ed09fe59064f05a81 2026-05-05 01:07:53,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:53,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9429ed09fe59064f05a81 HTTP/1.1" 200 None 2026-05-05 01:07:53,306 - lava_callback - DEBUG - Processing node 69f9429ed09fe59064f05a81: baseline-arm64-pengutronix 2026-05-05 01:07:53,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:53,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:53,558 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f9429ed09fe59064f05a81/log.txt.gz 2026-05-05 01:07:53,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:53,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:53,776 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f9429ed09fe59064f05a81/lava-logs.yaml 2026-05-05 01:07:53,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:07:53,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:07:53,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f9429ed09fe59064f05a81/lava_callback.json.gz 2026-05-05 01:07:53,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:53,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9429ed09fe59064f05a81 HTTP/1.1" 200 None 2026-05-05 01:07:53,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:53,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:07:53,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:07:53,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9429ed09fe59064f05a81 HTTP/1.1" 200 None 2026-05-05 01:07:53,961 - lava_callback - INFO - Completed processing callback for node 69f9429ed09fe59064f05a81 INFO: 172.20.0.1:50992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40880 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:08:09,343 - lava_callback - INFO - Received callback for node 69f942a1d09fe59064f05a89 INFO: 172.20.0.1:40892 - "POST /node/69f942a1d09fe59064f05a89 HTTP/1.0" 202 Accepted 2026-05-05 01:08:09,351 - lava_callback - INFO - Starting async job submit for node 69f942a1d09fe59064f05a89 2026-05-05 01:08:09,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:09,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a1d09fe59064f05a89 HTTP/1.1" 200 None 2026-05-05 01:08:09,386 - lava_callback - DEBUG - Processing node 69f942a1d09fe59064f05a89: baseline-arm64-pengutronix 2026-05-05 01:08:09,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:09,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:09,703 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f942a1d09fe59064f05a89/log.txt.gz 2026-05-05 01:08:09,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:09,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:09,969 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f942a1d09fe59064f05a89/lava-logs.yaml 2026-05-05 01:08:09,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:10,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:10,086 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f942a1d09fe59064f05a89/lava_callback.json.gz 2026-05-05 01:08:10,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:10,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a1d09fe59064f05a89 HTTP/1.1" 200 None 2026-05-05 01:08:10,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:10,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:08:10,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:10,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f942a1d09fe59064f05a89 HTTP/1.1" 200 None 2026-05-05 01:08:10,166 - lava_callback - INFO - Completed processing callback for node 69f942a1d09fe59064f05a89 2026-05-05 01:08:13,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:13,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:36972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36978 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:08:24,708 - lava_callback - INFO - Received callback for node 69f69ff21f903fec0b0ded80 INFO: 172.20.0.1:53690 - "POST /node/69f69ff21f903fec0b0ded80 HTTP/1.0" 202 Accepted 2026-05-05 01:08:24,714 - lava_callback - INFO - Starting async job submit for node 69f69ff21f903fec0b0ded80 2026-05-05 01:08:24,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:24,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69ff21f903fec0b0ded80 HTTP/1.1" 200 None 2026-05-05 01:08:24,729 - lava_callback - DEBUG - Processing node 69f69ff21f903fec0b0ded80: baseline-arm-clabbe 2026-05-05 01:08:24,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f69ff21f903fec0b0ded80 2026-05-05 01:08:24,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:24,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f69ff21f903fec0b0ded80 HTTP/1.1" 200 None 2026-05-05 01:08:24,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:24,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 01:08:24,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:24,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f69ff21f903fec0b0ded80 HTTP/1.1" 200 None 2026-05-05 01:08:24,776 - lava_callback - ERROR - Error processing callback for node 69f69ff21f903fec0b0ded80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51196 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:08:43,831 - lava_callback - INFO - Received callback for node 69f9429ad09fe59064f05a75 INFO: 172.20.0.1:45334 - "POST /node/69f9429ad09fe59064f05a75 HTTP/1.0" 202 Accepted 2026-05-05 01:08:43,837 - lava_callback - INFO - Starting async job submit for node 69f9429ad09fe59064f05a75 2026-05-05 01:08:43,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:43,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9429ad09fe59064f05a75 HTTP/1.1" 200 None 2026-05-05 01:08:43,875 - lava_callback - DEBUG - Processing node 69f9429ad09fe59064f05a75: baseline-arm64-clabbe 2026-05-05 01:08:44,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:44,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:44,206 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69f9429ad09fe59064f05a75/log.txt.gz 2026-05-05 01:08:44,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:44,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:44,653 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69f9429ad09fe59064f05a75/lava-logs.yaml 2026-05-05 01:08:44,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:44,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:44,782 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69f9429ad09fe59064f05a75/lava_callback.json.gz 2026-05-05 01:08:44,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:44,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9429ad09fe59064f05a75 HTTP/1.1" 200 None 2026-05-05 01:08:44,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:44,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:08:44,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:44,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9429ad09fe59064f05a75 HTTP/1.1" 200 None 2026-05-05 01:08:44,856 - lava_callback - INFO - Completed processing callback for node 69f9429ad09fe59064f05a75 INFO: 172.20.0.1:45336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51740 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:08:58,822 - lava_callback - INFO - Received callback for node 69f942a0d09fe59064f05a84 INFO: 172.20.0.1:51750 - "POST /node/69f942a0d09fe59064f05a84 HTTP/1.0" 202 Accepted 2026-05-05 01:08:58,829 - lava_callback - INFO - Starting async job submit for node 69f942a0d09fe59064f05a84 2026-05-05 01:08:58,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:58,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a0d09fe59064f05a84 HTTP/1.1" 200 None 2026-05-05 01:08:58,870 - lava_callback - DEBUG - Processing node 69f942a0d09fe59064f05a84: baseline-arm64-pengutronix 2026-05-05 01:08:59,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:59,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:59,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f942a0d09fe59064f05a84/log.txt.gz 2026-05-05 01:08:59,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:59,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:59,477 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f942a0d09fe59064f05a84/lava-logs.yaml 2026-05-05 01:08:59,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:08:59,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:08:59,612 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f942a0d09fe59064f05a84/lava_callback.json.gz 2026-05-05 01:08:59,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:59,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a0d09fe59064f05a84 HTTP/1.1" 200 None 2026-05-05 01:08:59,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:59,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:08:59,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:08:59,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f942a0d09fe59064f05a84 HTTP/1.1" 200 None 2026-05-05 01:08:59,779 - lava_callback - INFO - Completed processing callback for node 69f942a0d09fe59064f05a84 INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:09:13,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:13,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57002 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:09:24,845 - lava_callback - INFO - Received callback for node 69f6a0381f903fec0b0deda0 INFO: 172.20.0.1:40566 - "POST /node/69f6a0381f903fec0b0deda0 HTTP/1.0" 202 Accepted 2026-05-05 01:09:24,851 - lava_callback - INFO - Starting async job submit for node 69f6a0381f903fec0b0deda0 2026-05-05 01:09:24,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:24,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a0381f903fec0b0deda0 HTTP/1.1" 200 None 2026-05-05 01:09:24,867 - lava_callback - DEBUG - Processing node 69f6a0381f903fec0b0deda0: baseline-arm-clabbe 2026-05-05 01:09:24,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a0381f903fec0b0deda0 2026-05-05 01:09:24,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:24,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a0381f903fec0b0deda0 HTTP/1.1" 200 None 2026-05-05 01:09:24,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:24,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 01:09:24,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:24,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a0381f903fec0b0deda0 HTTP/1.1" 200 None 2026-05-05 01:09:24,917 - lava_callback - ERROR - Error processing callback for node 69f6a0381f903fec0b0deda0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42656 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:09:38,278 - lava_callback - INFO - Received callback for node 69f94263d09fe59064f0599c INFO: 172.20.0.1:42664 - "POST /node/69f94263d09fe59064f0599c HTTP/1.0" 202 Accepted 2026-05-05 01:09:38,285 - lava_callback - INFO - Starting async job submit for node 69f94263d09fe59064f0599c 2026-05-05 01:09:38,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:38,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94263d09fe59064f0599c HTTP/1.1" 200 None 2026-05-05 01:09:38,317 - lava_callback - DEBUG - Processing node 69f94263d09fe59064f0599c: baseline-arm64 2026-05-05 01:09:38,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:09:38,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:09:38,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94263d09fe59064f0599c/log.txt.gz 2026-05-05 01:09:39,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:09:39,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:09:39,262 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94263d09fe59064f0599c/lava-logs.yaml 2026-05-05 01:09:39,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:09:39,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:09:39,409 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94263d09fe59064f0599c/lava_callback.json.gz 2026-05-05 01:09:39,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:39,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94263d09fe59064f0599c HTTP/1.1" 200 None 2026-05-05 01:09:39,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:39,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:09:39,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:39,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94263d09fe59064f0599c HTTP/1.1" 200 None 2026-05-05 01:09:39,550 - lava_callback - INFO - Completed processing callback for node 69f94263d09fe59064f0599c 2026-05-05 01:09:43,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:09:44,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52598 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:12:25,797 - lava_callback - INFO - Received callback for node 69f942a3d09fe59064f05a8f INFO: 172.20.0.1:39436 - "POST /node/69f942a3d09fe59064f05a8f HTTP/1.0" 202 Accepted 2026-05-05 01:12:25,811 - lava_callback - INFO - Starting async job submit for node 69f942a3d09fe59064f05a8f 2026-05-05 01:12:25,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:12:25,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a3d09fe59064f05a8f HTTP/1.1" 200 None 2026-05-05 01:12:25,900 - lava_callback - DEBUG - Processing node 69f942a3d09fe59064f05a8f: baseline-arm64-foundriesio INFO: 172.20.0.1:39448 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:12:27,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:12:27,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:12:27,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f942a3d09fe59064f05a8f/log.txt.gz 2026-05-05 01:12:28,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:12:28,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:12:28,679 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f942a3d09fe59064f05a8f/lava-logs.yaml INFO: 172.20.0.1:39454 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:12:28,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:12:28,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:12:28,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f942a3d09fe59064f05a8f/lava_callback.json.gz DEBUG: Setup failed for 69f942a3d09fe59064f05a8f. Transitting job node result: incomplete -> fail 2026-05-05 01:12:28,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:12:28,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f942a3d09fe59064f05a8f HTTP/1.1" 200 None 2026-05-05 01:12:28,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:12:28,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:12:28,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:12:29,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f942a3d09fe59064f05a8f HTTP/1.1" 200 None 2026-05-05 01:12:29,123 - lava_callback - INFO - Completed processing callback for node 69f942a3d09fe59064f05a8f INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:12:44,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:12:44,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:13:04,981 - lava_callback - INFO - Received callback for node 69f6a1101f903fec0b0dee1e INFO: 172.20.0.1:40538 - "POST /node/69f6a1101f903fec0b0dee1e HTTP/1.0" 202 Accepted 2026-05-05 01:13:04,991 - lava_callback - INFO - Starting async job submit for node 69f6a1101f903fec0b0dee1e 2026-05-05 01:13:04,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:05,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a1101f903fec0b0dee1e HTTP/1.1" 200 None 2026-05-05 01:13:05,010 - lava_callback - DEBUG - Processing node 69f6a1101f903fec0b0dee1e: baseline-arm-clabbe 2026-05-05 01:13:05,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a1101f903fec0b0dee1e 2026-05-05 01:13:05,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:05,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a1101f903fec0b0dee1e HTTP/1.1" 200 None 2026-05-05 01:13:05,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:05,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 01:13:05,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:05,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a1101f903fec0b0dee1e HTTP/1.1" 200 None 2026-05-05 01:13:05,105 - lava_callback - ERROR - Error processing callback for node 69f6a1101f903fec0b0dee1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44486 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:13:37,715 - lava_callback - INFO - Received callback for node 69f94270d09fe59064f059e5 INFO: 172.20.0.1:44492 - "POST /node/69f94270d09fe59064f059e5 HTTP/1.0" 202 Accepted 2026-05-05 01:13:37,727 - lava_callback - INFO - Starting async job submit for node 69f94270d09fe59064f059e5 2026-05-05 01:13:37,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:37,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94270d09fe59064f059e5 HTTP/1.1" 200 None 2026-05-05 01:13:37,791 - lava_callback - DEBUG - Processing node 69f94270d09fe59064f059e5: baseline-arm64 2026-05-05 01:13:38,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:13:38,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:13:38,399 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94270d09fe59064f059e5/log.txt.gz 2026-05-05 01:13:38,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:13:38,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:13:38,914 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94270d09fe59064f059e5/lava-logs.yaml 2026-05-05 01:13:38,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:13:39,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:13:39,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94270d09fe59064f059e5/lava_callback.json.gz DEBUG: Setup failed for 69f94270d09fe59064f059e5. Transitting job node result: incomplete -> fail 2026-05-05 01:13:39,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:39,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94270d09fe59064f059e5 HTTP/1.1" 200 None 2026-05-05 01:13:39,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:39,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:13:39,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:39,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94270d09fe59064f059e5 HTTP/1.1" 200 None 2026-05-05 01:13:39,300 - lava_callback - INFO - Completed processing callback for node 69f94270d09fe59064f059e5 2026-05-05 01:13:44,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:13:44,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38620 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:14:02,788 - lava_callback - INFO - Received callback for node 69f94272d09fe59064f059eb INFO: 172.20.0.1:46062 - "POST /node/69f94272d09fe59064f059eb HTTP/1.0" 202 Accepted 2026-05-05 01:14:02,795 - lava_callback - INFO - Starting async job submit for node 69f94272d09fe59064f059eb 2026-05-05 01:14:02,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:02,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94272d09fe59064f059eb HTTP/1.1" 200 None 2026-05-05 01:14:02,839 - lava_callback - DEBUG - Processing node 69f94272d09fe59064f059eb: baseline-nfs-arm64 2026-05-05 01:14:03,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:14:03,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:14:03,402 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94272d09fe59064f059eb/log.txt.gz 2026-05-05 01:14:03,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:14:03,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:14:03,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94272d09fe59064f059eb/lava-logs.yaml 2026-05-05 01:14:03,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:14:03,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:14:03,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94272d09fe59064f059eb/lava_callback.json.gz 2026-05-05 01:14:03,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:03,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94272d09fe59064f059eb HTTP/1.1" 200 None 2026-05-05 01:14:03,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:03,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:14:03,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:03,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94272d09fe59064f059eb HTTP/1.1" 200 None 2026-05-05 01:14:03,998 - lava_callback - INFO - Completed processing callback for node 69f94272d09fe59064f059eb INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:14:14,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:14,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37538 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:14:47,101 - lava_callback - INFO - Received callback for node 69f6a1811f903fec0b0dee5f INFO: 172.20.0.1:37542 - "POST /node/69f6a1811f903fec0b0dee5f HTTP/1.0" 202 Accepted 2026-05-05 01:14:47,107 - lava_callback - INFO - Starting async job submit for node 69f6a1811f903fec0b0dee5f 2026-05-05 01:14:47,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:47,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a1811f903fec0b0dee5f HTTP/1.1" 200 None 2026-05-05 01:14:47,125 - lava_callback - DEBUG - Processing node 69f6a1811f903fec0b0dee5f: baseline-arm64-baylibre 2026-05-05 01:14:47,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a1811f903fec0b0dee5f 2026-05-05 01:14:47,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:47,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a1811f903fec0b0dee5f HTTP/1.1" 200 None 2026-05-05 01:14:47,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:47,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:14:47,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:14:47,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a1811f903fec0b0dee5f HTTP/1.1" 200 None 2026-05-05 01:14:47,182 - lava_callback - ERROR - Error processing callback for node 69f6a1811f903fec0b0dee5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53348 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:17:31,386 - lava_callback - INFO - Received callback for node 69f93a80d09fe59064f0547b INFO: 172.20.0.1:40518 - "POST /node/69f93a80d09fe59064f0547b HTTP/1.0" 202 Accepted 2026-05-05 01:17:31,392 - lava_callback - INFO - Starting async job submit for node 69f93a80d09fe59064f0547b 2026-05-05 01:17:31,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:17:31,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a80d09fe59064f0547b HTTP/1.1" 200 None 2026-05-05 01:17:31,433 - lava_callback - DEBUG - Processing node 69f93a80d09fe59064f0547b: baseline-x86 2026-05-05 01:17:31,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:17:31,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:17:31,918 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a80d09fe59064f0547b/log.txt.gz 2026-05-05 01:17:32,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:17:32,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:17:32,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a80d09fe59064f0547b/lava-logs.yaml 2026-05-05 01:17:32,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:17:32,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:17:32,431 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69f93a80d09fe59064f0547b/lava_callback.json.gz 2026-05-05 01:17:32,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:17:32,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93a80d09fe59064f0547b HTTP/1.1" 200 None 2026-05-05 01:17:32,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:17:32,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9372ed09fe59064f02d1d HTTP/1.1" 200 None 2026-05-05 01:17:32,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:17:32,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93a80d09fe59064f0547b HTTP/1.1" 200 None 2026-05-05 01:17:32,510 - lava_callback - INFO - Completed processing callback for node 69f93a80d09fe59064f0547b INFO: 172.20.0.1:40530 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:17:44,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:17:44,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:18:25,224 - lava_callback - INFO - Received callback for node 69f6a2591f903fec0b0deee9 INFO: 172.20.0.1:57364 - "POST /node/69f6a2591f903fec0b0deee9 HTTP/1.0" 202 Accepted 2026-05-05 01:18:25,230 - lava_callback - INFO - Starting async job submit for node 69f6a2591f903fec0b0deee9 2026-05-05 01:18:25,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:18:25,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a2591f903fec0b0deee9 HTTP/1.1" 200 None 2026-05-05 01:18:25,245 - lava_callback - DEBUG - Processing node 69f6a2591f903fec0b0deee9: baseline-arm64-clabbe 2026-05-05 01:18:25,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a2591f903fec0b0deee9 2026-05-05 01:18:25,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:18:25,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a2591f903fec0b0deee9 HTTP/1.1" 200 None 2026-05-05 01:18:25,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:18:25,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:18:25,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:18:25,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a2591f903fec0b0deee9 HTTP/1.1" 200 None 2026-05-05 01:18:25,304 - lava_callback - ERROR - Error processing callback for node 69f6a2591f903fec0b0deee9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49042 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:19:45,219 - lava_callback - INFO - Received callback for node 69f6a2a31f903fec0b0def1d INFO: 172.20.0.1:53742 - "POST /node/69f6a2a31f903fec0b0def1d HTTP/1.0" 202 Accepted 2026-05-05 01:19:45,225 - lava_callback - INFO - Starting async job submit for node 69f6a2a31f903fec0b0def1d 2026-05-05 01:19:45,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:45,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a2a31f903fec0b0def1d HTTP/1.1" 200 None 2026-05-05 01:19:45,240 - lava_callback - DEBUG - Processing node 69f6a2a31f903fec0b0def1d: baseline-arm64-clabbe 2026-05-05 01:19:45,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a2a31f903fec0b0def1d 2026-05-05 01:19:45,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:45,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a2a31f903fec0b0def1d HTTP/1.1" 200 None 2026-05-05 01:19:45,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:45,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:19:45,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:45,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a2a31f903fec0b0def1d HTTP/1.1" 200 None 2026-05-05 01:19:45,302 - lava_callback - ERROR - Error processing callback for node 69f6a2a31f903fec0b0def1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53748 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:19:55,297 - lava_callback - INFO - Received callback for node 69f94590d09fe59064f05d1c INFO: 172.20.0.1:49240 - "POST /node/69f94590d09fe59064f05d1c HTTP/1.0" 202 Accepted 2026-05-05 01:19:55,304 - lava_callback - INFO - Starting async job submit for node 69f94590d09fe59064f05d1c 2026-05-05 01:19:55,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:55,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94590d09fe59064f05d1c HTTP/1.1" 200 None 2026-05-05 01:19:55,338 - lava_callback - DEBUG - Processing node 69f94590d09fe59064f05d1c: baseline-arm64 2026-05-05 01:19:55,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:19:55,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:19:55,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94590d09fe59064f05d1c/log.txt.gz 2026-05-05 01:19:55,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:19:56,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:19:56,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94590d09fe59064f05d1c/lava-logs.yaml 2026-05-05 01:19:56,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:19:56,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:19:56,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94590d09fe59064f05d1c/lava_callback.json.gz 2026-05-05 01:19:56,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:56,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94590d09fe59064f05d1c HTTP/1.1" 200 None 2026-05-05 01:19:56,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:56,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:19:56,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:19:56,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94590d09fe59064f05d1c HTTP/1.1" 200 None 2026-05-05 01:19:56,259 - lava_callback - INFO - Completed processing callback for node 69f94590d09fe59064f05d1c INFO: 172.20.0.1:49252 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:20:00,556 - lava_callback - INFO - Received callback for node 69f94594d09fe59064f05d1f INFO: 172.20.0.1:41196 - "POST /node/69f94594d09fe59064f05d1f HTTP/1.0" 202 Accepted 2026-05-05 01:20:00,563 - lava_callback - INFO - Starting async job submit for node 69f94594d09fe59064f05d1f 2026-05-05 01:20:00,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:20:00,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94594d09fe59064f05d1f HTTP/1.1" 200 None 2026-05-05 01:20:00,594 - lava_callback - DEBUG - Processing node 69f94594d09fe59064f05d1f: baseline-arm64 2026-05-05 01:20:00,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:20:00,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:20:00,869 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94594d09fe59064f05d1f/log.txt.gz 2026-05-05 01:20:00,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:20:01,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:20:01,099 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94594d09fe59064f05d1f/lava-logs.yaml 2026-05-05 01:20:01,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:20:01,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:20:01,234 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94594d09fe59064f05d1f/lava_callback.json.gz 2026-05-05 01:20:01,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:20:01,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94594d09fe59064f05d1f HTTP/1.1" 200 None 2026-05-05 01:20:01,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:20:01,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:20:01,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:20:01,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94594d09fe59064f05d1f HTTP/1.1" 200 None 2026-05-05 01:20:01,320 - lava_callback - INFO - Completed processing callback for node 69f94594d09fe59064f05d1f INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:20:14,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:20:14,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:21:25,301 - lava_callback - INFO - Received callback for node 69f6a3021f903fec0b0def55 INFO: 172.20.0.1:37678 - "POST /node/69f6a3021f903fec0b0def55 HTTP/1.0" 202 Accepted 2026-05-05 01:21:25,306 - lava_callback - INFO - Starting async job submit for node 69f6a3021f903fec0b0def55 2026-05-05 01:21:25,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:21:25,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a3021f903fec0b0def55 HTTP/1.1" 200 None 2026-05-05 01:21:25,322 - lava_callback - DEBUG - Processing node 69f6a3021f903fec0b0def55: baseline-arm-clabbe 2026-05-05 01:21:25,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a3021f903fec0b0def55 2026-05-05 01:21:25,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:21:25,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a3021f903fec0b0def55 HTTP/1.1" 200 None 2026-05-05 01:21:25,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:21:25,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 01:21:25,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:21:25,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a3021f903fec0b0def55 HTTP/1.1" 200 None 2026-05-05 01:21:25,372 - lava_callback - ERROR - Error processing callback for node 69f6a3021f903fec0b0def55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38308 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:22:25,139 - lava_callback - INFO - Received callback for node 69f9463ed09fe59064f05d8a INFO: 172.20.0.1:56272 - "POST /node/69f9463ed09fe59064f05d8a HTTP/1.0" 202 Accepted 2026-05-05 01:22:25,144 - lava_callback - INFO - Starting async job submit for node 69f9463ed09fe59064f05d8a 2026-05-05 01:22:25,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:25,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9463ed09fe59064f05d8a HTTP/1.1" 200 None 2026-05-05 01:22:25,160 - lava_callback - DEBUG - Processing node 69f9463ed09fe59064f05d8a: baseline-nfs-arm64 2026-05-05 01:22:25,161 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:22:25,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:22:25,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:22:25,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9463ed09fe59064f05d8a/lava-logs.yaml 2026-05-05 01:22:25,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:22:25,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:22:25,676 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9463ed09fe59064f05d8a/lava_callback.json.gz 2026-05-05 01:22:25,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:25,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9463ed09fe59064f05d8a HTTP/1.1" 200 None 2026-05-05 01:22:25,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:25,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:22:25,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:25,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9463ed09fe59064f05d8a HTTP/1.1" 200 None 2026-05-05 01:22:25,740 - lava_callback - INFO - Completed processing callback for node 69f9463ed09fe59064f05d8a INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:22:44,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:44,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 01:22:45,371 - lava_callback - INFO - Received callback for node 69f6a35a1f903fec0b0def85 INFO: 172.20.0.1:39026 - "POST /node/69f6a35a1f903fec0b0def85 HTTP/1.0" 202 Accepted 2026-05-05 01:22:45,377 - lava_callback - INFO - Starting async job submit for node 69f6a35a1f903fec0b0def85 2026-05-05 01:22:45,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:45,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a35a1f903fec0b0def85 HTTP/1.1" 200 None 2026-05-05 01:22:45,392 - lava_callback - DEBUG - Processing node 69f6a35a1f903fec0b0def85: baseline-arm64-clabbe 2026-05-05 01:22:45,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a35a1f903fec0b0def85 2026-05-05 01:22:45,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:45,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a35a1f903fec0b0def85 HTTP/1.1" 200 None 2026-05-05 01:22:45,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:45,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:22:45,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:22:45,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a35a1f903fec0b0def85 HTTP/1.1" 200 None 2026-05-05 01:22:45,452 - lava_callback - ERROR - Error processing callback for node 69f6a35a1f903fec0b0def85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37316 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 01:24:15,729 - lava_callback - INFO - Received callback for node 69f94689d09fe59064f05dc0 INFO: 172.20.0.1:37330 - "POST /node/69f94689d09fe59064f05dc0 HTTP/1.0" 202 Accepted 2026-05-05 01:24:15,735 - lava_callback - INFO - Starting async job submit for node 69f94689d09fe59064f05dc0 2026-05-05 01:24:15,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:24:15,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94689d09fe59064f05dc0 HTTP/1.1" 200 None 2026-05-05 01:24:15,801 - lava_callback - DEBUG - Processing node 69f94689d09fe59064f05dc0: baseline-arm64 2026-05-05 01:24:15,960 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:24:16,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:24:16,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:24:16,173 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94689d09fe59064f05dc0/lava-logs.yaml 2026-05-05 01:24:16,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:24:16,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:24:16,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94689d09fe59064f05dc0/lava_callback.json.gz 2026-05-05 01:24:16,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:24:16,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94689d09fe59064f05dc0 HTTP/1.1" 200 None 2026-05-05 01:24:16,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:37338 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:24:16,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:24:16,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:24:16,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94689d09fe59064f05dc0 HTTP/1.1" 200 None 2026-05-05 01:24:16,597 - lava_callback - INFO - Completed processing callback for node 69f94689d09fe59064f05dc0 INFO: 172.20.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60076 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:24:44,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:24:44,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60936 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:25:35,604 - lava_callback - INFO - Received callback for node 69f946dcd09fe59064f05df0 INFO: 172.20.0.1:44532 - "POST /node/69f946dcd09fe59064f05df0 HTTP/1.0" 202 Accepted 2026-05-05 01:25:35,610 - lava_callback - INFO - Starting async job submit for node 69f946dcd09fe59064f05df0 2026-05-05 01:25:35,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:25:35,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f946dcd09fe59064f05df0 HTTP/1.1" 200 None 2026-05-05 01:25:35,661 - lava_callback - DEBUG - Processing node 69f946dcd09fe59064f05df0: baseline-arm64 2026-05-05 01:25:35,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:25:36,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:25:36,084 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f946dcd09fe59064f05df0/log.txt.gz 2026-05-05 01:25:36,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:25:36,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:25:36,295 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f946dcd09fe59064f05df0/lava-logs.yaml 2026-05-05 01:25:36,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:25:36,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:25:36,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f946dcd09fe59064f05df0/lava_callback.json.gz 2026-05-05 01:25:36,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:25:36,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f946dcd09fe59064f05df0 HTTP/1.1" 200 None 2026-05-05 01:25:36,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:25:36,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:25:36,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:25:36,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f946dcd09fe59064f05df0 HTTP/1.1" 200 None INFO: 172.20.0.1:44534 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:25:36,551 - lava_callback - INFO - Completed processing callback for node 69f946dcd09fe59064f05df0 2026-05-05 01:25:44,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:25:44,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37118 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:26:07,400 - lava_callback - INFO - Received callback for node 69f94274d09fe59064f059f1 INFO: 172.20.0.1:37126 - "POST /node/69f94274d09fe59064f059f1 HTTP/1.0" 202 Accepted 2026-05-05 01:26:07,408 - lava_callback - INFO - Starting async job submit for node 69f94274d09fe59064f059f1 2026-05-05 01:26:07,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:07,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94274d09fe59064f059f1 HTTP/1.1" 200 None 2026-05-05 01:26:07,468 - lava_callback - DEBUG - Processing node 69f94274d09fe59064f059f1: baseline-nfs-arm64 2026-05-05 01:26:08,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:26:08,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:26:08,754 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94274d09fe59064f059f1/log.txt.gz 2026-05-05 01:26:09,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:26:09,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:26:09,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94274d09fe59064f059f1/lava-logs.yaml 2026-05-05 01:26:09,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:26:09,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:26:09,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94274d09fe59064f059f1/lava_callback.json.gz DEBUG: Setup failed for 69f94274d09fe59064f059f1. Transitting job node result: incomplete -> fail 2026-05-05 01:26:09,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:09,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94274d09fe59064f059f1 HTTP/1.1" 200 None 2026-05-05 01:26:09,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:10,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:26:10,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:10,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94274d09fe59064f059f1 HTTP/1.1" 200 None 2026-05-05 01:26:10,119 - lava_callback - INFO - Completed processing callback for node 69f94274d09fe59064f059f1 2026-05-05 01:26:14,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:14,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:26:45,506 - lava_callback - INFO - Received callback for node 69f6a44c1f903fec0b0df018 INFO: 172.20.0.1:56280 - "POST /node/69f6a44c1f903fec0b0df018 HTTP/1.0" 202 Accepted 2026-05-05 01:26:45,511 - lava_callback - INFO - Starting async job submit for node 69f6a44c1f903fec0b0df018 2026-05-05 01:26:45,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:45,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a44c1f903fec0b0df018 HTTP/1.1" 200 None 2026-05-05 01:26:45,526 - lava_callback - DEBUG - Processing node 69f6a44c1f903fec0b0df018: baseline-arm-clabbe 2026-05-05 01:26:45,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a44c1f903fec0b0df018 2026-05-05 01:26:45,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:45,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a44c1f903fec0b0df018 HTTP/1.1" 200 None 2026-05-05 01:26:45,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:45,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 01:26:45,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:26:45,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a44c1f903fec0b0df018 HTTP/1.1" 200 None 2026-05-05 01:26:45,575 - lava_callback - ERROR - Error processing callback for node 69f6a44c1f903fec0b0df018: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:28:10,976 - lava_callback - INFO - Received callback for node 69f9476ed09fe59064f05e50 INFO: 172.20.0.1:51316 - "POST /node/69f9476ed09fe59064f05e50 HTTP/1.0" 202 Accepted 2026-05-05 01:28:10,982 - lava_callback - INFO - Starting async job submit for node 69f9476ed09fe59064f05e50 2026-05-05 01:28:11,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:28:11,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9476ed09fe59064f05e50 HTTP/1.1" 200 None 2026-05-05 01:28:11,018 - lava_callback - DEBUG - Processing node 69f9476ed09fe59064f05e50: baseline-arm64-pengutronix 2026-05-05 01:28:11,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:28:11,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:28:11,496 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f9476ed09fe59064f05e50/log.txt.gz 2026-05-05 01:28:11,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:28:11,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:28:11,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f9476ed09fe59064f05e50/lava-logs.yaml 2026-05-05 01:28:11,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:28:11,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:28:11,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69f9476ed09fe59064f05e50/lava_callback.json.gz 2026-05-05 01:28:11,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:28:11,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9476ed09fe59064f05e50 HTTP/1.1" 200 None 2026-05-05 01:28:11,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:28:11,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:28:11,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:28:12,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9476ed09fe59064f05e50 HTTP/1.1" 200 None 2026-05-05 01:28:12,006 - lava_callback - INFO - Completed processing callback for node 69f9476ed09fe59064f05e50 2026-05-05 01:28:14,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:28:14,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49160 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:30:05,724 - lava_callback - INFO - Received callback for node 69f6a50f1f903fec0b0df07c INFO: 172.20.0.1:36226 - "POST /node/69f6a50f1f903fec0b0df07c HTTP/1.0" 202 Accepted 2026-05-05 01:30:05,729 - lava_callback - INFO - Starting async job submit for node 69f6a50f1f903fec0b0df07c 2026-05-05 01:30:05,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:30:05,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a50f1f903fec0b0df07c HTTP/1.1" 200 None 2026-05-05 01:30:05,747 - lava_callback - DEBUG - Processing node 69f6a50f1f903fec0b0df07c: baseline-arm64-clabbe 2026-05-05 01:30:05,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a50f1f903fec0b0df07c 2026-05-05 01:30:05,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:30:05,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a50f1f903fec0b0df07c HTTP/1.1" 200 None 2026-05-05 01:30:05,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:30:05,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:30:05,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:30:05,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a50f1f903fec0b0df07c HTTP/1.1" 200 None 2026-05-05 01:30:05,798 - lava_callback - ERROR - Error processing callback for node 69f6a50f1f903fec0b0df07c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37534 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:31:32,471 - lava_callback - INFO - Received callback for node 69f94499d09fe59064f05caf INFO: 172.20.0.1:53772 - "POST /node/69f94499d09fe59064f05caf HTTP/1.0" 202 Accepted 2026-05-05 01:31:32,478 - lava_callback - INFO - Starting async job submit for node 69f94499d09fe59064f05caf 2026-05-05 01:31:32,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:31:32,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94499d09fe59064f05caf HTTP/1.1" 200 None 2026-05-05 01:31:32,544 - lava_callback - DEBUG - Processing node 69f94499d09fe59064f05caf: baseline-arm 2026-05-05 01:31:33,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:31:33,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:31:33,268 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f94499d09fe59064f05caf/log.txt.gz 2026-05-05 01:31:33,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:31:33,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:31:33,831 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f94499d09fe59064f05caf/lava-logs.yaml 2026-05-05 01:31:33,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:31:34,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:31:34,139 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f94499d09fe59064f05caf/lava_callback.json.gz DEBUG: Setup failed for 69f94499d09fe59064f05caf. Transitting job node result: incomplete -> fail 2026-05-05 01:31:34,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:31:34,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94499d09fe59064f05caf HTTP/1.1" 200 None 2026-05-05 01:31:34,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:31:34,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 01:31:34,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:31:34,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94499d09fe59064f05caf HTTP/1.1" 200 None 2026-05-05 01:31:34,387 - lava_callback - INFO - Completed processing callback for node 69f94499d09fe59064f05caf INFO: 172.20.0.1:53776 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:31:44,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:31:44,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50674 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:32:10,571 - lava_callback - INFO - Received callback for node 69f947d1d09fe59064f05e96 INFO: 172.20.0.1:44620 - "POST /node/69f947d1d09fe59064f05e96 HTTP/1.0" 202 Accepted 2026-05-05 01:32:10,577 - lava_callback - INFO - Starting async job submit for node 69f947d1d09fe59064f05e96 2026-05-05 01:32:10,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:10,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f947d1d09fe59064f05e96 HTTP/1.1" 200 None 2026-05-05 01:32:10,609 - lava_callback - DEBUG - Processing node 69f947d1d09fe59064f05e96: baseline-arm64 2026-05-05 01:32:10,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:32:11,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:32:11,089 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f947d1d09fe59064f05e96/log.txt.gz 2026-05-05 01:32:11,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:32:11,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:32:11,427 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f947d1d09fe59064f05e96/lava-logs.yaml 2026-05-05 01:32:11,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:32:11,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:32:11,546 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f947d1d09fe59064f05e96/lava_callback.json.gz 2026-05-05 01:32:11,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:11,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f947d1d09fe59064f05e96 HTTP/1.1" 200 None 2026-05-05 01:32:11,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:11,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:32:11,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:11,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f947d1d09fe59064f05e96 HTTP/1.1" 200 None 2026-05-05 01:32:11,670 - lava_callback - INFO - Completed processing callback for node 69f947d1d09fe59064f05e96 2026-05-05 01:32:14,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:14,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44638 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:32:25,737 - lava_callback - INFO - Received callback for node 69f6a5a01f903fec0b0df0d1 INFO: 172.20.0.1:45950 - "POST /node/69f6a5a01f903fec0b0df0d1 HTTP/1.0" 202 Accepted 2026-05-05 01:32:25,745 - lava_callback - INFO - Starting async job submit for node 69f6a5a01f903fec0b0df0d1 2026-05-05 01:32:25,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:25,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a5a01f903fec0b0df0d1 HTTP/1.1" 200 None 2026-05-05 01:32:25,762 - lava_callback - DEBUG - Processing node 69f6a5a01f903fec0b0df0d1: baseline-arm64-clabbe 2026-05-05 01:32:25,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a5a01f903fec0b0df0d1 2026-05-05 01:32:25,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:25,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a5a01f903fec0b0df0d1 HTTP/1.1" 200 None 2026-05-05 01:32:25,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:25,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:32:25,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:32:25,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a5a01f903fec0b0df0d1 HTTP/1.1" 200 None 2026-05-05 01:32:25,811 - lava_callback - ERROR - Error processing callback for node 69f6a5a01f903fec0b0df0d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36306 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:35:05,890 - lava_callback - INFO - Received callback for node 69f6a6341f903fec0b0df12b INFO: 172.20.0.1:45670 - "POST /node/69f6a6341f903fec0b0df12b HTTP/1.0" 202 Accepted 2026-05-05 01:35:05,895 - lava_callback - INFO - Starting async job submit for node 69f6a6341f903fec0b0df12b 2026-05-05 01:35:05,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:05,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a6341f903fec0b0df12b HTTP/1.1" 200 None 2026-05-05 01:35:05,911 - lava_callback - DEBUG - Processing node 69f6a6341f903fec0b0df12b: baseline-arm64-clabbe 2026-05-05 01:35:05,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a6341f903fec0b0df12b 2026-05-05 01:35:05,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:05,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a6341f903fec0b0df12b HTTP/1.1" 200 None 2026-05-05 01:35:05,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:05,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:35:05,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:05,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a6341f903fec0b0df12b HTTP/1.1" 200 None 2026-05-05 01:35:05,957 - lava_callback - ERROR - Error processing callback for node 69f6a6341f903fec0b0df12b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:35:25,615 - lava_callback - INFO - Received callback for node 69f9494dd09fe59064f05f83 INFO: 172.20.0.1:58988 - "POST /node/69f9494dd09fe59064f05f83 HTTP/1.0" 202 Accepted 2026-05-05 01:35:25,621 - lava_callback - INFO - Starting async job submit for node 69f9494dd09fe59064f05f83 2026-05-05 01:35:25,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:25,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9494dd09fe59064f05f83 HTTP/1.1" 200 None 2026-05-05 01:35:25,637 - lava_callback - DEBUG - Processing node 69f9494dd09fe59064f05f83: baseline-nfs-arm64 2026-05-05 01:35:25,639 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:35:25,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:35:25,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:35:25,902 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9494dd09fe59064f05f83/lava-logs.yaml 2026-05-05 01:35:25,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:35:26,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:35:26,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9494dd09fe59064f05f83/lava_callback.json.gz 2026-05-05 01:35:26,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:26,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9494dd09fe59064f05f83 HTTP/1.1" 200 None 2026-05-05 01:35:26,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:26,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:35:26,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:26,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9494dd09fe59064f05f83 HTTP/1.1" 200 None 2026-05-05 01:35:26,076 - lava_callback - INFO - Completed processing callback for node 69f9494dd09fe59064f05f83 INFO: 172.20.0.1:58992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40588 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:35:44,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:35:44,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60328 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:36:05,885 - lava_callback - INFO - Received callback for node 69f6a6711f903fec0b0df149 INFO: 172.20.0.1:45524 - "POST /node/69f6a6711f903fec0b0df149 HTTP/1.0" 202 Accepted 2026-05-05 01:36:05,891 - lava_callback - INFO - Starting async job submit for node 69f6a6711f903fec0b0df149 2026-05-05 01:36:05,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:05,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a6711f903fec0b0df149 HTTP/1.1" 200 None 2026-05-05 01:36:05,907 - lava_callback - DEBUG - Processing node 69f6a6711f903fec0b0df149: baseline-arm-clabbe 2026-05-05 01:36:05,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a6711f903fec0b0df149 2026-05-05 01:36:05,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:05,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a6711f903fec0b0df149 HTTP/1.1" 200 None 2026-05-05 01:36:05,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:05,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 01:36:05,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:05,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a6711f903fec0b0df149 HTTP/1.1" 200 None 2026-05-05 01:36:05,967 - lava_callback - ERROR - Error processing callback for node 69f6a6711f903fec0b0df149: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42032 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 01:36:16,000 - lava_callback - INFO - Received callback for node 69f9483cd09fe59064f05ec7 INFO: 172.20.0.1:42046 - "POST /node/69f9483cd09fe59064f05ec7 HTTP/1.0" 202 Accepted 2026-05-05 01:36:16,010 - lava_callback - INFO - Starting async job submit for node 69f9483cd09fe59064f05ec7 2026-05-05 01:36:16,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:16,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9483cd09fe59064f05ec7 HTTP/1.1" 200 None 2026-05-05 01:36:16,087 - lava_callback - DEBUG - Processing node 69f9483cd09fe59064f05ec7: baseline-arm64-foundriesio INFO: 172.20.0.1:42050 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:36:17,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:36:17,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:36:17,358 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f9483cd09fe59064f05ec7/log.txt.gz 2026-05-05 01:36:18,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:36:18,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:36:18,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f9483cd09fe59064f05ec7/lava-logs.yaml 2026-05-05 01:36:18,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:36:18,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:36:18,421 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f9483cd09fe59064f05ec7/lava_callback.json.gz DEBUG: Setup failed for 69f9483cd09fe59064f05ec7. Transitting job node result: incomplete -> fail 2026-05-05 01:36:18,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:18,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9483cd09fe59064f05ec7 HTTP/1.1" 200 None 2026-05-05 01:36:18,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:18,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:36:18,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:18,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9483cd09fe59064f05ec7 HTTP/1.1" 200 None 2026-05-05 01:36:18,696 - lava_callback - INFO - Completed processing callback for node 69f9483cd09fe59064f05ec7 INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56512 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:36:44,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:44,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 01:36:45,872 - lava_callback - INFO - Received callback for node 69f6a6a31f903fec0b0df162 INFO: 172.20.0.1:44292 - "POST /node/69f6a6a31f903fec0b0df162 HTTP/1.0" 202 Accepted 2026-05-05 01:36:45,877 - lava_callback - INFO - Starting async job submit for node 69f6a6a31f903fec0b0df162 2026-05-05 01:36:45,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:45,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a6a31f903fec0b0df162 HTTP/1.1" 200 None 2026-05-05 01:36:45,892 - lava_callback - DEBUG - Processing node 69f6a6a31f903fec0b0df162: baseline-arm64-clabbe 2026-05-05 01:36:45,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a6a31f903fec0b0df162 2026-05-05 01:36:45,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:45,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a6a31f903fec0b0df162 HTTP/1.1" 200 None 2026-05-05 01:36:45,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:45,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:36:45,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:36:45,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a6a31f903fec0b0df162 HTTP/1.1" 200 None 2026-05-05 01:36:45,937 - lava_callback - ERROR - Error processing callback for node 69f6a6a31f903fec0b0df162: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46588 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 01:37:16,316 - lava_callback - INFO - Received callback for node 69f94998d09fe59064f05fc6 INFO: 172.20.0.1:46592 - "POST /node/69f94998d09fe59064f05fc6 HTTP/1.0" 202 Accepted 2026-05-05 01:37:16,323 - lava_callback - INFO - Starting async job submit for node 69f94998d09fe59064f05fc6 2026-05-05 01:37:16,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:37:16,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94998d09fe59064f05fc6 HTTP/1.1" 200 None 2026-05-05 01:37:16,386 - lava_callback - DEBUG - Processing node 69f94998d09fe59064f05fc6: baseline-arm64 2026-05-05 01:37:16,519 - lava_callback - WARNING - Failed to upload log INFO: 172.20.0.1:46594 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:37:16,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:37:16,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:37:16,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94998d09fe59064f05fc6/lava-logs.yaml 2026-05-05 01:37:16,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:37:16,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:37:16,878 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94998d09fe59064f05fc6/lava_callback.json.gz 2026-05-05 01:37:16,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:37:16,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94998d09fe59064f05fc6 HTTP/1.1" 200 None 2026-05-05 01:37:16,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:37:16,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:37:16,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:37:16,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94998d09fe59064f05fc6 HTTP/1.1" 200 None 2026-05-05 01:37:17,025 - lava_callback - INFO - Completed processing callback for node 69f94998d09fe59064f05fc6 INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:37:44,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:37:44,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51176 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:41:26,188 - lava_callback - INFO - Received callback for node 69f6a7bb1f903fec0b0df205 INFO: 172.20.0.1:36208 - "POST /node/69f6a7bb1f903fec0b0df205 HTTP/1.0" 202 Accepted 2026-05-05 01:41:26,193 - lava_callback - INFO - Starting async job submit for node 69f6a7bb1f903fec0b0df205 2026-05-05 01:41:26,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:26,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a7bb1f903fec0b0df205 HTTP/1.1" 200 None 2026-05-05 01:41:26,215 - lava_callback - DEBUG - Processing node 69f6a7bb1f903fec0b0df205: baseline-arm-clabbe 2026-05-05 01:41:26,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a7bb1f903fec0b0df205 2026-05-05 01:41:26,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:26,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a7bb1f903fec0b0df205 HTTP/1.1" 200 None 2026-05-05 01:41:26,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:26,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 01:41:26,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:26,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a7bb1f903fec0b0df205 HTTP/1.1" 200 None 2026-05-05 01:41:26,264 - lava_callback - ERROR - Error processing callback for node 69f6a7bb1f903fec0b0df205: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:41:30,153 - lava_callback - INFO - Received callback for node 69f948a3d09fe59064f05f16 INFO: 172.20.0.1:47840 - "POST /node/69f948a3d09fe59064f05f16 HTTP/1.0" 202 Accepted 2026-05-05 01:41:30,160 - lava_callback - INFO - Starting async job submit for node 69f948a3d09fe59064f05f16 2026-05-05 01:41:30,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:30,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f948a3d09fe59064f05f16 HTTP/1.1" 200 None 2026-05-05 01:41:30,217 - lava_callback - DEBUG - Processing node 69f948a3d09fe59064f05f16: baseline-arm64 2026-05-05 01:41:30,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:41:30,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:41:30,868 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f948a3d09fe59064f05f16/log.txt.gz 2026-05-05 01:41:31,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:41:31,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:41:31,281 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f948a3d09fe59064f05f16/lava-logs.yaml 2026-05-05 01:41:31,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:41:31,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:41:31,617 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f948a3d09fe59064f05f16/lava_callback.json.gz DEBUG: Setup failed for 69f948a3d09fe59064f05f16. Transitting job node result: incomplete -> fail 2026-05-05 01:41:31,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:31,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f948a3d09fe59064f05f16 HTTP/1.1" 200 None 2026-05-05 01:41:31,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:31,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:41:31,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:31,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f948a3d09fe59064f05f16 HTTP/1.1" 200 None 2026-05-05 01:41:31,805 - lava_callback - INFO - Completed processing callback for node 69f948a3d09fe59064f05f16 INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:41:44,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:44,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 01:41:44,545 - lava_callback - INFO - Received callback for node 69f949ead09fe59064f05ff6 INFO: 172.20.0.1:54748 - "POST /node/69f949ead09fe59064f05ff6 HTTP/1.0" 202 Accepted 2026-05-05 01:41:44,552 - lava_callback - INFO - Starting async job submit for node 69f949ead09fe59064f05ff6 2026-05-05 01:41:44,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:44,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f949ead09fe59064f05ff6 HTTP/1.1" 200 None 2026-05-05 01:41:44,596 - lava_callback - DEBUG - Processing node 69f949ead09fe59064f05ff6: baseline-arm64 2026-05-05 01:41:44,816 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:41:44,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:41:45,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:41:45,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f949ead09fe59064f05ff6/lava-logs.yaml 2026-05-05 01:41:45,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:41:45,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:41:45,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f949ead09fe59064f05ff6/lava_callback.json.gz 2026-05-05 01:41:45,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:45,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f949ead09fe59064f05ff6 HTTP/1.1" 200 None 2026-05-05 01:41:45,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:45,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:41:45,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:41:45,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f949ead09fe59064f05ff6 HTTP/1.1" 200 None 2026-05-05 01:41:45,322 - lava_callback - INFO - Completed processing callback for node 69f949ead09fe59064f05ff6 INFO: 172.20.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36558 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:42:14,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:42:14,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52428 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:42:26,108 - lava_callback - INFO - Received callback for node 69f6a7f11f903fec0b0df224 INFO: 172.20.0.1:60956 - "POST /node/69f6a7f11f903fec0b0df224 HTTP/1.0" 202 Accepted 2026-05-05 01:42:26,113 - lava_callback - INFO - Starting async job submit for node 69f6a7f11f903fec0b0df224 2026-05-05 01:42:26,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:42:26,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a7f11f903fec0b0df224 HTTP/1.1" 200 None 2026-05-05 01:42:26,128 - lava_callback - DEBUG - Processing node 69f6a7f11f903fec0b0df224: baseline-arm64-clabbe 2026-05-05 01:42:26,129 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a7f11f903fec0b0df224 2026-05-05 01:42:26,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:42:26,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a7f11f903fec0b0df224 HTTP/1.1" 200 None 2026-05-05 01:42:26,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:42:26,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:42:26,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:42:26,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a7f11f903fec0b0df224 HTTP/1.1" 200 None 2026-05-05 01:42:26,212 - lava_callback - ERROR - Error processing callback for node 69f6a7f11f903fec0b0df224: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47522 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:44:20,890 - lava_callback - INFO - Received callback for node 69f94b5fd09fe59064f060e9 INFO: 172.20.0.1:56250 - "POST /node/69f94b5fd09fe59064f060e9 HTTP/1.0" 202 Accepted 2026-05-05 01:44:20,896 - lava_callback - INFO - Starting async job submit for node 69f94b5fd09fe59064f060e9 2026-05-05 01:44:20,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:20,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94b5fd09fe59064f060e9 HTTP/1.1" 200 None 2026-05-05 01:44:20,912 - lava_callback - DEBUG - Processing node 69f94b5fd09fe59064f060e9: baseline-nfs-arm64 2026-05-05 01:44:20,914 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:44:20,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:44:21,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:44:21,016 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94b5fd09fe59064f060e9/lava-logs.yaml 2026-05-05 01:44:21,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:44:21,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:44:21,119 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94b5fd09fe59064f060e9/lava_callback.json.gz 2026-05-05 01:44:21,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:21,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94b5fd09fe59064f060e9 HTTP/1.1" 200 None 2026-05-05 01:44:21,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:21,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:44:21,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:21,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94b5fd09fe59064f060e9 HTTP/1.1" 200 None 2026-05-05 01:44:21,179 - lava_callback - INFO - Completed processing callback for node 69f94b5fd09fe59064f060e9 INFO: 172.20.0.1:56262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47124 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:44:44,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:44,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:44:53,160 - lava_callback - INFO - Received callback for node 69f94ab8d09fe59064f0606f INFO: 172.20.0.1:39472 - "POST /node/69f94ab8d09fe59064f0606f HTTP/1.0" 202 Accepted 2026-05-05 01:44:53,167 - lava_callback - INFO - Starting async job submit for node 69f94ab8d09fe59064f0606f 2026-05-05 01:44:53,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:53,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94ab8d09fe59064f0606f HTTP/1.1" 200 None 2026-05-05 01:44:53,207 - lava_callback - DEBUG - Processing node 69f94ab8d09fe59064f0606f: baseline-arm64 2026-05-05 01:44:53,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:44:53,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:44:53,639 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94ab8d09fe59064f0606f/log.txt.gz 2026-05-05 01:44:53,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:44:54,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:44:54,171 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94ab8d09fe59064f0606f/lava-logs.yaml 2026-05-05 01:44:54,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:44:54,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:44:54,283 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94ab8d09fe59064f0606f/lava_callback.json.gz 2026-05-05 01:44:54,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:54,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94ab8d09fe59064f0606f HTTP/1.1" 200 None 2026-05-05 01:44:54,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:54,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:44:54,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:44:54,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94ab8d09fe59064f0606f HTTP/1.1" 200 None 2026-05-05 01:44:54,408 - lava_callback - INFO - Completed processing callback for node 69f94ab8d09fe59064f0606f INFO: 172.20.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:45:08,756 - lava_callback - INFO - Received callback for node 69f6a8a01f903fec0b0df277 INFO: 172.20.0.1:37242 - "POST /node/69f6a8a01f903fec0b0df277 HTTP/1.0" 202 Accepted 2026-05-05 01:45:08,762 - lava_callback - INFO - Starting async job submit for node 69f6a8a01f903fec0b0df277 2026-05-05 01:45:08,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:45:08,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a8a01f903fec0b0df277 HTTP/1.1" 200 None 2026-05-05 01:45:08,777 - lava_callback - DEBUG - Processing node 69f6a8a01f903fec0b0df277: baseline-arm64-baylibre 2026-05-05 01:45:08,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a8a01f903fec0b0df277 2026-05-05 01:45:08,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:45:08,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a8a01f903fec0b0df277 HTTP/1.1" 200 None 2026-05-05 01:45:08,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:45:08,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 01:45:08,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:45:08,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a8a01f903fec0b0df277 HTTP/1.1" 200 None 2026-05-05 01:45:08,911 - lava_callback - ERROR - Error processing callback for node 69f6a8a01f903fec0b0df277: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 01:45:14,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:45:14,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34842 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:47:06,277 - lava_callback - INFO - Received callback for node 69f6a9071f903fec0b0df2a9 INFO: 172.20.0.1:53108 - "POST /node/69f6a9071f903fec0b0df2a9 HTTP/1.0" 202 Accepted 2026-05-05 01:47:06,283 - lava_callback - INFO - Starting async job submit for node 69f6a9071f903fec0b0df2a9 2026-05-05 01:47:06,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:06,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a9071f903fec0b0df2a9 HTTP/1.1" 200 None 2026-05-05 01:47:06,297 - lava_callback - DEBUG - Processing node 69f6a9071f903fec0b0df2a9: baseline-arm-clabbe 2026-05-05 01:47:06,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a9071f903fec0b0df2a9 2026-05-05 01:47:06,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:06,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a9071f903fec0b0df2a9 HTTP/1.1" 200 None 2026-05-05 01:47:06,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:06,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 01:47:06,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:06,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a9071f903fec0b0df2a9 HTTP/1.1" 200 None 2026-05-05 01:47:06,353 - lava_callback - ERROR - Error processing callback for node 69f6a9071f903fec0b0df2a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56772 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:47:35,749 - lava_callback - INFO - Received callback for node 69f94bfcd09fe59064f0614c INFO: 172.20.0.1:38152 - "POST /node/69f94bfcd09fe59064f0614c HTTP/1.0" 202 Accepted 2026-05-05 01:47:35,755 - lava_callback - INFO - Starting async job submit for node 69f94bfcd09fe59064f0614c 2026-05-05 01:47:35,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:35,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94bfcd09fe59064f0614c HTTP/1.1" 200 None 2026-05-05 01:47:35,805 - lava_callback - DEBUG - Processing node 69f94bfcd09fe59064f0614c: baseline-arm64 2026-05-05 01:47:35,938 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:47:36,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:47:36,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:47:36,320 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94bfcd09fe59064f0614c/lava-logs.yaml 2026-05-05 01:47:36,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:47:36,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:47:36,431 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94bfcd09fe59064f0614c/lava_callback.json.gz 2026-05-05 01:47:36,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:36,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94bfcd09fe59064f0614c HTTP/1.1" 200 None 2026-05-05 01:47:36,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:36,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:47:36,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:38168 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:47:36,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94bfcd09fe59064f0614c HTTP/1.1" 200 None 2026-05-05 01:47:36,608 - lava_callback - INFO - Completed processing callback for node 69f94bfcd09fe59064f0614c 2026-05-05 01:47:44,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:44,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51128 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:47:48,768 - lava_callback - INFO - Received callback for node 69f6a9321f903fec0b0df2c0 INFO: 172.20.0.1:51130 - "POST /node/69f6a9321f903fec0b0df2c0 HTTP/1.0" 202 Accepted 2026-05-05 01:47:48,774 - lava_callback - INFO - Starting async job submit for node 69f6a9321f903fec0b0df2c0 2026-05-05 01:47:48,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:48,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a9321f903fec0b0df2c0 HTTP/1.1" 200 None 2026-05-05 01:47:48,789 - lava_callback - DEBUG - Processing node 69f6a9321f903fec0b0df2c0: baseline-arm64-baylibre 2026-05-05 01:47:48,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a9321f903fec0b0df2c0 2026-05-05 01:47:48,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:48,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a9321f903fec0b0df2c0 HTTP/1.1" 200 None 2026-05-05 01:47:48,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:48,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 01:47:48,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:47:48,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a9321f903fec0b0df2c0 HTTP/1.1" 200 None 2026-05-05 01:47:48,856 - lava_callback - ERROR - Error processing callback for node 69f6a9321f903fec0b0df2c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58264 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:48:41,461 - lava_callback - INFO - Received callback for node 69f94bc4d09fe59064f06126 INFO: 172.20.0.1:43890 - "POST /node/69f94bc4d09fe59064f06126 HTTP/1.0" 202 Accepted 2026-05-05 01:48:41,469 - lava_callback - INFO - Starting async job submit for node 69f94bc4d09fe59064f06126 2026-05-05 01:48:41,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:48:41,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94bc4d09fe59064f06126 HTTP/1.1" 200 None 2026-05-05 01:48:41,521 - lava_callback - DEBUG - Processing node 69f94bc4d09fe59064f06126: baseline-arm64-foundriesio 2026-05-05 01:48:42,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:48:42,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:48:42,427 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f94bc4d09fe59064f06126/log.txt.gz 2026-05-05 01:48:42,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:48:43,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:48:43,113 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f94bc4d09fe59064f06126/lava-logs.yaml 2026-05-05 01:48:43,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:48:43,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:48:43,305 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f94bc4d09fe59064f06126/lava_callback.json.gz DEBUG: 69f94bc4d09fe59064f06126 Transitting job node result: pass -> incomplete 2026-05-05 01:48:43,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:48:43,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94bc4d09fe59064f06126 HTTP/1.1" 200 None 2026-05-05 01:48:43,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:48:43,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:48:43,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:48:43,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94bc4d09fe59064f06126 HTTP/1.1" 200 None 2026-05-05 01:48:43,402 - lava_callback - INFO - Completed processing callback for node 69f94bc4d09fe59064f06126 2026-05-05 01:48:44,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:48:44,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54458 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:50:06,444 - lava_callback - INFO - Received callback for node 69f6a9ca1f903fec0b0df322 INFO: 172.20.0.1:38320 - "POST /node/69f6a9ca1f903fec0b0df322 HTTP/1.0" 202 Accepted 2026-05-05 01:50:06,449 - lava_callback - INFO - Starting async job submit for node 69f6a9ca1f903fec0b0df322 2026-05-05 01:50:06,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:06,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a9ca1f903fec0b0df322 HTTP/1.1" 200 None 2026-05-05 01:50:06,465 - lava_callback - DEBUG - Processing node 69f6a9ca1f903fec0b0df322: baseline-arm64-clabbe 2026-05-05 01:50:06,465 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6a9ca1f903fec0b0df322 2026-05-05 01:50:06,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:06,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6a9ca1f903fec0b0df322 HTTP/1.1" 200 None 2026-05-05 01:50:06,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:06,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:50:06,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:06,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6a9ca1f903fec0b0df322 HTTP/1.1" 200 None 2026-05-05 01:50:06,515 - lava_callback - ERROR - Error processing callback for node 69f6a9ca1f903fec0b0df322: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38322 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:50:07,279 - lava_callback - INFO - Received callback for node 69f9401bd09fe59064f0585b INFO: 172.20.0.1:38324 - "POST /node/69f9401bd09fe59064f0585b HTTP/1.0" 202 Accepted 2026-05-05 01:50:07,285 - lava_callback - INFO - Starting async job submit for node 69f9401bd09fe59064f0585b 2026-05-05 01:50:07,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:07,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9401bd09fe59064f0585b HTTP/1.1" 200 None 2026-05-05 01:50:07,322 - lava_callback - DEBUG - Processing node 69f9401bd09fe59064f0585b: baseline-arm 2026-05-05 01:50:07,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:50:07,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:50:07,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f9401bd09fe59064f0585b/log.txt.gz 2026-05-05 01:50:07,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:50:07,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:50:07,911 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f9401bd09fe59064f0585b/lava-logs.yaml 2026-05-05 01:50:07,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:50:08,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:50:08,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f9401bd09fe59064f0585b/lava_callback.json.gz 2026-05-05 01:50:08,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:08,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9401bd09fe59064f0585b HTTP/1.1" 200 None 2026-05-05 01:50:08,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:08,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93708d09fe59064f02cf1 HTTP/1.1" 200 None 2026-05-05 01:50:08,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:08,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9401bd09fe59064f0585b HTTP/1.1" 200 None 2026-05-05 01:50:08,120 - lava_callback - INFO - Completed processing callback for node 69f9401bd09fe59064f0585b 2026-05-05 01:50:14,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:14,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52278 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:50:59,896 - lava_callback - INFO - Received callback for node 69f94ca1d09fe59064f061c2 INFO: 172.20.0.1:52760 - "POST /node/69f94ca1d09fe59064f061c2 HTTP/1.0" 202 Accepted 2026-05-05 01:50:59,903 - lava_callback - INFO - Starting async job submit for node 69f94ca1d09fe59064f061c2 2026-05-05 01:50:59,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:50:59,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94ca1d09fe59064f061c2 HTTP/1.1" 200 None 2026-05-05 01:50:59,949 - lava_callback - DEBUG - Processing node 69f94ca1d09fe59064f061c2: baseline-arm64 2026-05-05 01:51:00,086 - lava_callback - WARNING - Failed to upload log 2026-05-05 01:51:00,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:51:00,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:51:00,306 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94ca1d09fe59064f061c2/lava-logs.yaml 2026-05-05 01:51:00,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:51:00,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:51:00,415 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94ca1d09fe59064f061c2/lava_callback.json.gz 2026-05-05 01:51:00,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:51:00,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94ca1d09fe59064f061c2 HTTP/1.1" 200 None 2026-05-05 01:51:00,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:51:00,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:51:00,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:51:00,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94ca1d09fe59064f061c2 HTTP/1.1" 200 None 2026-05-05 01:51:00,628 - lava_callback - INFO - Completed processing callback for node 69f94ca1d09fe59064f061c2 INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:51:14,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:51:14,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48490 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:52:09,397 - lava_callback - INFO - Received callback for node 69f6aa461f903fec0b0df352 INFO: 172.20.0.1:48504 - "POST /node/69f6aa461f903fec0b0df352 HTTP/1.0" 202 Accepted 2026-05-05 01:52:09,402 - lava_callback - INFO - Starting async job submit for node 69f6aa461f903fec0b0df352 2026-05-05 01:52:09,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:52:09,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6aa461f903fec0b0df352 HTTP/1.1" 200 None 2026-05-05 01:52:09,419 - lava_callback - DEBUG - Processing node 69f6aa461f903fec0b0df352: baseline-arm64-baylibre 2026-05-05 01:52:09,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6aa461f903fec0b0df352 2026-05-05 01:52:09,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:52:09,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6aa461f903fec0b0df352 HTTP/1.1" 200 None 2026-05-05 01:52:09,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:52:09,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 01:52:09,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:52:09,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6aa461f903fec0b0df352 HTTP/1.1" 200 None 2026-05-05 01:52:09,465 - lava_callback - ERROR - Error processing callback for node 69f6aa461f903fec0b0df352: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35978 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:53:07,864 - lava_callback - INFO - Received callback for node 69f9428fd09fe59064f05a57 INFO: 172.20.0.1:35988 - "POST /node/69f9428fd09fe59064f05a57 HTTP/1.0" 202 Accepted 2026-05-05 01:53:07,870 - lava_callback - INFO - Starting async job submit for node 69f9428fd09fe59064f05a57 2026-05-05 01:53:07,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:53:07,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9428fd09fe59064f05a57 HTTP/1.1" 200 None 2026-05-05 01:53:07,916 - lava_callback - DEBUG - Processing node 69f9428fd09fe59064f05a57: baseline-arm64-broonie 2026-05-05 01:53:08,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:53:08,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:53:08,294 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428fd09fe59064f05a57/log.txt.gz 2026-05-05 01:53:08,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:53:08,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:53:08,620 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428fd09fe59064f05a57/lava-logs.yaml 2026-05-05 01:53:08,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:53:08,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:53:08,746 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428fd09fe59064f05a57/lava_callback.json.gz 2026-05-05 01:53:08,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:53:08,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9428fd09fe59064f05a57 HTTP/1.1" 200 None 2026-05-05 01:53:08,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:53:08,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:53:08,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:53:08,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9428fd09fe59064f05a57 HTTP/1.1" 200 None 2026-05-05 01:53:08,843 - lava_callback - INFO - Completed processing callback for node 69f9428fd09fe59064f05a57 2026-05-05 01:53:14,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:53:14,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57814 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:54:34,820 - lava_callback - INFO - Received callback for node 69f94cf5d09fe59064f061ef INFO: 172.20.0.1:33246 - "POST /node/69f94cf5d09fe59064f061ef HTTP/1.0" 202 Accepted 2026-05-05 01:54:34,826 - lava_callback - INFO - Starting async job submit for node 69f94cf5d09fe59064f061ef 2026-05-05 01:54:34,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:34,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94cf5d09fe59064f061ef HTTP/1.1" 200 None 2026-05-05 01:54:34,857 - lava_callback - DEBUG - Processing node 69f94cf5d09fe59064f061ef: baseline-arm64 2026-05-05 01:54:35,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:54:35,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:54:35,363 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94cf5d09fe59064f061ef/log.txt.gz 2026-05-05 01:54:35,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:54:35,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:54:35,885 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94cf5d09fe59064f061ef/lava-logs.yaml 2026-05-05 01:54:35,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:54:36,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:54:36,023 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94cf5d09fe59064f061ef/lava_callback.json.gz 2026-05-05 01:54:36,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:36,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94cf5d09fe59064f061ef HTTP/1.1" 200 None 2026-05-05 01:54:36,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:36,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:54:36,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:36,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94cf5d09fe59064f061ef HTTP/1.1" 200 None 2026-05-05 01:54:36,169 - lava_callback - INFO - Completed processing callback for node 69f94cf5d09fe59064f061ef INFO: 172.20.0.1:33260 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:54:44,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:44,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35202 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:54:49,237 - lava_callback - INFO - Received callback for node 69f6aadb1f903fec0b0df3a3 INFO: 172.20.0.1:35206 - "POST /node/69f6aadb1f903fec0b0df3a3 HTTP/1.0" 202 Accepted 2026-05-05 01:54:49,242 - lava_callback - INFO - Starting async job submit for node 69f6aadb1f903fec0b0df3a3 2026-05-05 01:54:49,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:49,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6aadb1f903fec0b0df3a3 HTTP/1.1" 200 None 2026-05-05 01:54:49,259 - lava_callback - DEBUG - Processing node 69f6aadb1f903fec0b0df3a3: baseline-arm64-baylibre 2026-05-05 01:54:49,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6aadb1f903fec0b0df3a3 2026-05-05 01:54:49,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:49,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6aadb1f903fec0b0df3a3 HTTP/1.1" 200 None 2026-05-05 01:54:49,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:49,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 01:54:49,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:54:49,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6aadb1f903fec0b0df3a3 HTTP/1.1" 200 None 2026-05-05 01:54:49,308 - lava_callback - ERROR - Error processing callback for node 69f6aadb1f903fec0b0df3a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:55:37,671 - lava_callback - INFO - Received callback for node 69f94c52d09fe59064f06184 INFO: 172.20.0.1:58566 - "POST /node/69f94c52d09fe59064f06184 HTTP/1.0" 202 Accepted 2026-05-05 01:55:37,678 - lava_callback - INFO - Starting async job submit for node 69f94c52d09fe59064f06184 2026-05-05 01:55:37,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:55:37,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94c52d09fe59064f06184 HTTP/1.1" 200 None 2026-05-05 01:55:37,728 - lava_callback - DEBUG - Processing node 69f94c52d09fe59064f06184: baseline-arm64 2026-05-05 01:55:38,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:55:38,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:55:38,212 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94c52d09fe59064f06184/log.txt.gz 2026-05-05 01:55:38,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:55:38,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:55:38,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94c52d09fe59064f06184/lava-logs.yaml 2026-05-05 01:55:38,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:55:38,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:55:38,754 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94c52d09fe59064f06184/lava_callback.json.gz DEBUG: Setup failed for 69f94c52d09fe59064f06184. Transitting job node result: incomplete -> fail 2026-05-05 01:55:38,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:55:38,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94c52d09fe59064f06184 HTTP/1.1" 200 None 2026-05-05 01:55:38,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:55:38,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:55:38,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:55:38,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94c52d09fe59064f06184 HTTP/1.1" 200 None 2026-05-05 01:55:38,945 - lava_callback - INFO - Completed processing callback for node 69f94c52d09fe59064f06184 2026-05-05 01:55:44,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:55:44,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57494 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:56:46,871 - lava_callback - INFO - Received callback for node 69f6ab4f1f903fec0b0df3e8 INFO: 172.20.0.1:57498 - "POST /node/69f6ab4f1f903fec0b0df3e8 HTTP/1.0" 202 Accepted 2026-05-05 01:56:46,876 - lava_callback - INFO - Starting async job submit for node 69f6ab4f1f903fec0b0df3e8 2026-05-05 01:56:46,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:56:46,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ab4f1f903fec0b0df3e8 HTTP/1.1" 200 None 2026-05-05 01:56:46,903 - lava_callback - DEBUG - Processing node 69f6ab4f1f903fec0b0df3e8: baseline-arm64-clabbe 2026-05-05 01:56:46,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ab4f1f903fec0b0df3e8 2026-05-05 01:56:46,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:56:46,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ab4f1f903fec0b0df3e8 HTTP/1.1" 200 None 2026-05-05 01:56:46,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:56:46,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 01:56:46,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:56:46,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ab4f1f903fec0b0df3e8 HTTP/1.1" 200 None 2026-05-05 01:56:46,955 - lava_callback - ERROR - Error processing callback for node 69f6ab4f1f903fec0b0df3e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46084 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:57:16,752 - lava_callback - INFO - Received callback for node 69f94a90d09fe59064f06059 INFO: 172.20.0.1:46090 - "POST /node/69f94a90d09fe59064f06059 HTTP/1.0" 202 Accepted 2026-05-05 01:57:16,759 - lava_callback - INFO - Starting async job submit for node 69f94a90d09fe59064f06059 2026-05-05 01:57:16,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:16,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94a90d09fe59064f06059 HTTP/1.1" 200 None 2026-05-05 01:57:16,826 - lava_callback - DEBUG - Processing node 69f94a90d09fe59064f06059: baseline-arm 2026-05-05 01:57:17,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:57:17,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:57:17,705 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f94a90d09fe59064f06059/log.txt.gz 2026-05-05 01:57:18,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:57:18,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:57:18,402 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f94a90d09fe59064f06059/lava-logs.yaml 2026-05-05 01:57:18,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:57:18,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:57:18,552 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f94a90d09fe59064f06059/lava_callback.json.gz DEBUG: Setup failed for 69f94a90d09fe59064f06059. Transitting job node result: incomplete -> fail 2026-05-05 01:57:18,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:18,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94a90d09fe59064f06059 HTTP/1.1" 200 None 2026-05-05 01:57:18,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:18,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 01:57:18,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:18,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94a90d09fe59064f06059 HTTP/1.1" 200 None 2026-05-05 01:57:18,837 - lava_callback - INFO - Completed processing callback for node 69f94a90d09fe59064f06059 INFO: 172.20.0.1:57460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54638 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:57:44,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:44,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53148 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:57:49,373 - lava_callback - INFO - Received callback for node 69f6ab941f903fec0b0df40a INFO: 172.20.0.1:53164 - "POST /node/69f6ab941f903fec0b0df40a HTTP/1.0" 202 Accepted 2026-05-05 01:57:49,379 - lava_callback - INFO - Starting async job submit for node 69f6ab941f903fec0b0df40a 2026-05-05 01:57:49,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:49,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ab941f903fec0b0df40a HTTP/1.1" 200 None 2026-05-05 01:57:49,394 - lava_callback - DEBUG - Processing node 69f6ab941f903fec0b0df40a: baseline-arm64-baylibre 2026-05-05 01:57:49,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ab941f903fec0b0df40a 2026-05-05 01:57:49,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:49,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ab941f903fec0b0df40a HTTP/1.1" 200 None 2026-05-05 01:57:49,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:49,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 01:57:49,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:57:49,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ab941f903fec0b0df40a HTTP/1.1" 200 None 2026-05-05 01:57:49,457 - lava_callback - ERROR - Error processing callback for node 69f6ab941f903fec0b0df40a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46012 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:59:25,348 - lava_callback - INFO - Received callback for node 69f94a43d09fe59064f06027 INFO: 172.20.0.1:36974 - "POST /node/69f94a43d09fe59064f06027 HTTP/1.0" 202 Accepted 2026-05-05 01:59:25,356 - lava_callback - INFO - Starting async job submit for node 69f94a43d09fe59064f06027 2026-05-05 01:59:25,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:25,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94a43d09fe59064f06027 HTTP/1.1" 200 None 2026-05-05 01:59:25,413 - lava_callback - DEBUG - Processing node 69f94a43d09fe59064f06027: baseline-nfs-arm64 INFO: 172.20.0.1:36986 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:59:26,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:59:26,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:59:26,826 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94a43d09fe59064f06027/log.txt.gz 2026-05-05 01:59:27,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:59:27,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:59:27,661 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94a43d09fe59064f06027/lava-logs.yaml 2026-05-05 01:59:27,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 01:59:27,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 01:59:27,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94a43d09fe59064f06027/lava_callback.json.gz DEBUG: Setup failed for 69f94a43d09fe59064f06027. Transitting job node result: incomplete -> fail 2026-05-05 01:59:27,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:27,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94a43d09fe59064f06027 HTTP/1.1" 200 None 2026-05-05 01:59:27,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:27,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 01:59:27,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:27,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94a43d09fe59064f06027 HTTP/1.1" 200 None 2026-05-05 01:59:28,059 - lava_callback - INFO - Completed processing callback for node 69f94a43d09fe59064f06027 INFO: 172.20.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35932 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:59:44,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:44,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK 2026-05-05 01:59:49,472 - lava_callback - INFO - Received callback for node 69f6ac001f903fec0b0df43b INFO: 172.20.0.1:33654 - "POST /node/69f6ac001f903fec0b0df43b HTTP/1.0" 202 Accepted 2026-05-05 01:59:49,478 - lava_callback - INFO - Starting async job submit for node 69f6ac001f903fec0b0df43b 2026-05-05 01:59:49,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:49,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ac001f903fec0b0df43b HTTP/1.1" 200 None 2026-05-05 01:59:49,492 - lava_callback - DEBUG - Processing node 69f6ac001f903fec0b0df43b: baseline-arm64-baylibre 2026-05-05 01:59:49,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ac001f903fec0b0df43b 2026-05-05 01:59:49,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:49,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ac001f903fec0b0df43b HTTP/1.1" 200 None 2026-05-05 01:59:49,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:49,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 01:59:49,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 01:59:49,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ac001f903fec0b0df43b HTTP/1.1" 200 None 2026-05-05 01:59:49,542 - lava_callback - ERROR - Error processing callback for node 69f6ac001f903fec0b0df43b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:00:51,060 - lava_callback - INFO - Received callback for node 69f94dfed09fe59064f062be INFO: 172.20.0.1:58144 - "POST /node/69f94dfed09fe59064f062be HTTP/1.0" 202 Accepted 2026-05-05 02:00:51,069 - lava_callback - INFO - Starting async job submit for node 69f94dfed09fe59064f062be 2026-05-05 02:00:51,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:00:51,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94dfed09fe59064f062be HTTP/1.1" 200 None 2026-05-05 02:00:51,139 - lava_callback - DEBUG - Processing node 69f94dfed09fe59064f062be: baseline-arm64-foundriesio 2026-05-05 02:00:52,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:00:52,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:00:52,605 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f94dfed09fe59064f062be/log.txt.gz 2026-05-05 02:00:53,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:00:53,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:00:53,532 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f94dfed09fe59064f062be/lava-logs.yaml 2026-05-05 02:00:53,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:00:53,737 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:00:53,739 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69f94dfed09fe59064f062be/lava_callback.json.gz DEBUG: Setup failed for 69f94dfed09fe59064f062be. Transitting job node result: incomplete -> fail 2026-05-05 02:00:53,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:00:53,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94dfed09fe59064f062be HTTP/1.1" 200 None 2026-05-05 02:00:53,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:00:53,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 02:00:53,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:00:53,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94dfed09fe59064f062be HTTP/1.1" 200 None 2026-05-05 02:00:54,016 - lava_callback - INFO - Completed processing callback for node 69f94dfed09fe59064f062be INFO: 172.20.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:01:06,864 - lava_callback - INFO - Received callback for node 69f6ac511f903fec0b0df470 INFO: 172.20.0.1:54074 - "POST /node/69f6ac511f903fec0b0df470 HTTP/1.0" 202 Accepted 2026-05-05 02:01:06,869 - lava_callback - INFO - Starting async job submit for node 69f6ac511f903fec0b0df470 2026-05-05 02:01:06,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:01:06,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ac511f903fec0b0df470 HTTP/1.1" 200 None 2026-05-05 02:01:06,884 - lava_callback - DEBUG - Processing node 69f6ac511f903fec0b0df470: baseline-arm64-clabbe 2026-05-05 02:01:06,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ac511f903fec0b0df470 2026-05-05 02:01:06,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:01:06,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ac511f903fec0b0df470 HTTP/1.1" 200 None 2026-05-05 02:01:06,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:01:06,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 02:01:06,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:01:06,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ac511f903fec0b0df470 HTTP/1.1" 200 None 2026-05-05 02:01:06,930 - lava_callback - ERROR - Error processing callback for node 69f6ac511f903fec0b0df470: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 02:01:14,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:01:14,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50682 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:02:26,870 - lava_callback - INFO - Received callback for node 69f6ac9c1f903fec0b0df4af INFO: 172.20.0.1:50690 - "POST /node/69f6ac9c1f903fec0b0df4af HTTP/1.0" 202 Accepted 2026-05-05 02:02:26,875 - lava_callback - INFO - Starting async job submit for node 69f6ac9c1f903fec0b0df4af 2026-05-05 02:02:26,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:02:26,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ac9c1f903fec0b0df4af HTTP/1.1" 200 None 2026-05-05 02:02:26,891 - lava_callback - DEBUG - Processing node 69f6ac9c1f903fec0b0df4af: baseline-arm64-clabbe 2026-05-05 02:02:26,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ac9c1f903fec0b0df4af 2026-05-05 02:02:26,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:02:26,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ac9c1f903fec0b0df4af HTTP/1.1" 200 None 2026-05-05 02:02:26,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:02:26,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 02:02:26,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:02:26,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ac9c1f903fec0b0df4af HTTP/1.1" 200 None 2026-05-05 02:02:26,937 - lava_callback - ERROR - Error processing callback for node 69f6ac9c1f903fec0b0df4af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:04:06,951 - lava_callback - INFO - Received callback for node 69f6ad071f903fec0b0df4db INFO: 172.20.0.1:59534 - "POST /node/69f6ad071f903fec0b0df4db HTTP/1.0" 202 Accepted 2026-05-05 02:04:06,957 - lava_callback - INFO - Starting async job submit for node 69f6ad071f903fec0b0df4db 2026-05-05 02:04:06,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:04:06,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ad071f903fec0b0df4db HTTP/1.1" 200 None 2026-05-05 02:04:06,972 - lava_callback - DEBUG - Processing node 69f6ad071f903fec0b0df4db: baseline-arm64-clabbe 2026-05-05 02:04:06,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ad071f903fec0b0df4db 2026-05-05 02:04:06,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:04:06,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ad071f903fec0b0df4db HTTP/1.1" 200 None 2026-05-05 02:04:06,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:04:07,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 02:04:07,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:04:07,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ad071f903fec0b0df4db HTTP/1.1" 200 None 2026-05-05 02:04:07,025 - lava_callback - ERROR - Error processing callback for node 69f6ad071f903fec0b0df4db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:05:27,075 - lava_callback - INFO - Received callback for node 69f6ad511f903fec0b0df50b INFO: 172.20.0.1:48358 - "POST /node/69f6ad511f903fec0b0df50b HTTP/1.0" 202 Accepted 2026-05-05 02:05:27,081 - lava_callback - INFO - Starting async job submit for node 69f6ad511f903fec0b0df50b 2026-05-05 02:05:27,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:05:27,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ad511f903fec0b0df50b HTTP/1.1" 200 None 2026-05-05 02:05:27,105 - lava_callback - DEBUG - Processing node 69f6ad511f903fec0b0df50b: baseline-arm64-clabbe 2026-05-05 02:05:27,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ad511f903fec0b0df50b 2026-05-05 02:05:27,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:05:27,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ad511f903fec0b0df50b HTTP/1.1" 200 None 2026-05-05 02:05:27,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:05:27,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 02:05:27,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:05:27,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ad511f903fec0b0df50b HTTP/1.1" 200 None 2026-05-05 02:05:27,167 - lava_callback - ERROR - Error processing callback for node 69f6ad511f903fec0b0df50b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36848 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:06:24,456 - lava_callback - INFO - Received callback for node 69f94eddd09fe59064f06394 INFO: 172.20.0.1:37844 - "POST /node/69f94eddd09fe59064f06394 HTTP/1.0" 202 Accepted 2026-05-05 02:06:24,463 - lava_callback - INFO - Starting async job submit for node 69f94eddd09fe59064f06394 2026-05-05 02:06:24,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:06:24,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94eddd09fe59064f06394 HTTP/1.1" 200 None 2026-05-05 02:06:24,518 - lava_callback - DEBUG - Processing node 69f94eddd09fe59064f06394: baseline-arm64 2026-05-05 02:06:24,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:06:25,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:06:25,026 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94eddd09fe59064f06394/log.txt.gz 2026-05-05 02:06:25,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:06:25,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:06:25,436 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94eddd09fe59064f06394/lava-logs.yaml 2026-05-05 02:06:25,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:06:25,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:06:25,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69f94eddd09fe59064f06394/lava_callback.json.gz DEBUG: Setup failed for 69f94eddd09fe59064f06394. Transitting job node result: incomplete -> fail 2026-05-05 02:06:25,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:06:25,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94eddd09fe59064f06394 HTTP/1.1" 200 None 2026-05-05 02:06:25,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:06:25,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 02:06:25,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:06:25,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94eddd09fe59064f06394 HTTP/1.1" 200 None 2026-05-05 02:06:25,983 - lava_callback - INFO - Completed processing callback for node 69f94eddd09fe59064f06394 INFO: 172.20.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39210 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:06:44,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:06:44,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53478 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:07:07,133 - lava_callback - INFO - Received callback for node 69f6adbe1f903fec0b0df537 INFO: 172.20.0.1:53480 - "POST /node/69f6adbe1f903fec0b0df537 HTTP/1.0" 202 Accepted 2026-05-05 02:07:07,138 - lava_callback - INFO - Starting async job submit for node 69f6adbe1f903fec0b0df537 2026-05-05 02:07:07,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:07,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6adbe1f903fec0b0df537 HTTP/1.1" 200 None 2026-05-05 02:07:07,155 - lava_callback - DEBUG - Processing node 69f6adbe1f903fec0b0df537: baseline-arm64-clabbe 2026-05-05 02:07:07,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6adbe1f903fec0b0df537 2026-05-05 02:07:07,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:07,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6adbe1f903fec0b0df537 HTTP/1.1" 200 None 2026-05-05 02:07:07,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:07,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 02:07:07,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:07,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6adbe1f903fec0b0df537 HTTP/1.1" 200 None 2026-05-05 02:07:07,208 - lava_callback - ERROR - Error processing callback for node 69f6adbe1f903fec0b0df537: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57688 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:07:47,385 - lava_callback - INFO - Received callback for node 69f9401dd09fe59064f0585e INFO: 172.20.0.1:57692 - "POST /node/69f9401dd09fe59064f0585e HTTP/1.0" 202 Accepted 2026-05-05 02:07:47,392 - lava_callback - INFO - Starting async job submit for node 69f9401dd09fe59064f0585e 2026-05-05 02:07:47,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:47,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9401dd09fe59064f0585e HTTP/1.1" 200 None 2026-05-05 02:07:47,467 - lava_callback - DEBUG - Processing node 69f9401dd09fe59064f0585e: ltp-smoke 2026-05-05 02:07:47,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:07:48,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:07:48,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f9401dd09fe59064f0585e/log.txt.gz 2026-05-05 02:07:48,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:07:48,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:07:48,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f9401dd09fe59064f0585e/lava-logs.yaml 2026-05-05 02:07:48,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:07:48,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:07:48,832 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f9401dd09fe59064f0585e/lava_callback.json.gz 2026-05-05 02:07:48,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:48,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9401dd09fe59064f0585e HTTP/1.1" 200 None 2026-05-05 02:07:48,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:48,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93708d09fe59064f02cf1 HTTP/1.1" 200 None 2026-05-05 02:07:48,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:07:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9401dd09fe59064f0585e HTTP/1.1" 200 None 2026-05-05 02:07:49,145 - lava_callback - INFO - Completed processing callback for node 69f9401dd09fe59064f0585e INFO: 172.20.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:08:14,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:08:14,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-05-05 02:08:14,878 - lava_callback - INFO - Received callback for node 69f93fa1d09fe59064f0582c INFO: 172.20.0.1:56070 - "POST /node/69f93fa1d09fe59064f0582c HTTP/1.0" 202 Accepted 2026-05-05 02:08:14,884 - lava_callback - INFO - Starting async job submit for node 69f93fa1d09fe59064f0582c 2026-05-05 02:08:14,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:08:14,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93fa1d09fe59064f0582c HTTP/1.1" 200 None 2026-05-05 02:08:14,920 - lava_callback - DEBUG - Processing node 69f93fa1d09fe59064f0582c: baseline-arm-broonie 2026-05-05 02:08:15,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56086 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 02:08:15,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:08:15,256 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93fa1d09fe59064f0582c/log.txt.gz 2026-05-05 02:08:15,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:08:15,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:08:15,509 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93fa1d09fe59064f0582c/lava-logs.yaml 2026-05-05 02:08:15,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:08:15,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:08:15,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69f93fa1d09fe59064f0582c/lava_callback.json.gz 2026-05-05 02:08:15,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:08:15,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93fa1d09fe59064f0582c HTTP/1.1" 200 None 2026-05-05 02:08:15,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:08:15,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 02:08:15,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:08:15,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93fa1d09fe59064f0582c HTTP/1.1" 200 None 2026-05-05 02:08:15,715 - lava_callback - INFO - Completed processing callback for node 69f93fa1d09fe59064f0582c INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54808 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:08:44,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:08:44,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47258 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:11:27,334 - lava_callback - INFO - Received callback for node 69f6aebf1f903fec0b0df5c1 INFO: 172.20.0.1:47266 - "POST /node/69f6aebf1f903fec0b0df5c1 HTTP/1.0" 202 Accepted 2026-05-05 02:11:27,340 - lava_callback - INFO - Starting async job submit for node 69f6aebf1f903fec0b0df5c1 2026-05-05 02:11:27,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:11:27,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6aebf1f903fec0b0df5c1 HTTP/1.1" 200 None 2026-05-05 02:11:27,355 - lava_callback - DEBUG - Processing node 69f6aebf1f903fec0b0df5c1: baseline-arm64-clabbe 2026-05-05 02:11:27,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6aebf1f903fec0b0df5c1 2026-05-05 02:11:27,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:11:27,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6aebf1f903fec0b0df5c1 HTTP/1.1" 200 None 2026-05-05 02:11:27,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:11:27,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 02:11:27,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:11:27,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6aebf1f903fec0b0df5c1 HTTP/1.1" 200 None 2026-05-05 02:11:27,431 - lava_callback - ERROR - Error processing callback for node 69f6aebf1f903fec0b0df5c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38296 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:13:07,373 - lava_callback - INFO - Received callback for node 69f6af2f1f903fec0b0df5f9 INFO: 172.20.0.1:38306 - "POST /node/69f6af2f1f903fec0b0df5f9 HTTP/1.0" 202 Accepted 2026-05-05 02:13:07,379 - lava_callback - INFO - Starting async job submit for node 69f6af2f1f903fec0b0df5f9 2026-05-05 02:13:07,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:13:07,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6af2f1f903fec0b0df5f9 HTTP/1.1" 200 None 2026-05-05 02:13:07,394 - lava_callback - DEBUG - Processing node 69f6af2f1f903fec0b0df5f9: baseline-arm64-clabbe 2026-05-05 02:13:07,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6af2f1f903fec0b0df5f9 2026-05-05 02:13:07,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:13:07,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6af2f1f903fec0b0df5f9 HTTP/1.1" 200 None 2026-05-05 02:13:07,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:13:07,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 02:13:07,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:13:07,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6af2f1f903fec0b0df5f9 HTTP/1.1" 200 None 2026-05-05 02:13:07,441 - lava_callback - ERROR - Error processing callback for node 69f6af2f1f903fec0b0df5f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55040 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:14:27,450 - lava_callback - INFO - Received callback for node 69f6af7b1f903fec0b0df618 INFO: 172.20.0.1:55044 - "POST /node/69f6af7b1f903fec0b0df618 HTTP/1.0" 202 Accepted 2026-05-05 02:14:27,456 - lava_callback - INFO - Starting async job submit for node 69f6af7b1f903fec0b0df618 2026-05-05 02:14:27,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:14:27,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6af7b1f903fec0b0df618 HTTP/1.1" 200 None 2026-05-05 02:14:27,477 - lava_callback - DEBUG - Processing node 69f6af7b1f903fec0b0df618: baseline-arm64-clabbe 2026-05-05 02:14:27,477 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6af7b1f903fec0b0df618 2026-05-05 02:14:27,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:14:27,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6af7b1f903fec0b0df618 HTTP/1.1" 200 None 2026-05-05 02:14:27,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:14:27,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343cee61e72f6b29d1b22 HTTP/1.1" 200 None 2026-05-05 02:14:27,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:14:27,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6af7b1f903fec0b0df618 HTTP/1.1" 200 None 2026-05-05 02:14:27,546 - lava_callback - ERROR - Error processing callback for node 69f6af7b1f903fec0b0df618: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54758 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:15:27,519 - lava_callback - INFO - Received callback for node 69f6afb81f903fec0b0df639 INFO: 172.20.0.1:54772 - "POST /node/69f6afb81f903fec0b0df639 HTTP/1.0" 202 Accepted 2026-05-05 02:15:27,525 - lava_callback - INFO - Starting async job submit for node 69f6afb81f903fec0b0df639 2026-05-05 02:15:27,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:15:27,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6afb81f903fec0b0df639 HTTP/1.1" 200 None 2026-05-05 02:15:27,548 - lava_callback - DEBUG - Processing node 69f6afb81f903fec0b0df639: baseline-arm-clabbe 2026-05-05 02:15:27,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6afb81f903fec0b0df639 2026-05-05 02:15:27,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:15:27,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6afb81f903fec0b0df639 HTTP/1.1" 200 None 2026-05-05 02:15:27,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:15:27,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:15:27,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:15:27,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6afb81f903fec0b0df639 HTTP/1.1" 200 None 2026-05-05 02:15:27,685 - lava_callback - ERROR - Error processing callback for node 69f6afb81f903fec0b0df639: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59456 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:17:49,082 - lava_callback - INFO - Received callback for node 69f94294d09fe59064f05a63 INFO: 172.20.0.1:59472 - "POST /node/69f94294d09fe59064f05a63 HTTP/1.0" 202 Accepted 2026-05-05 02:17:49,129 - lava_callback - INFO - Starting async job submit for node 69f94294d09fe59064f05a63 2026-05-05 02:17:49,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:17:49,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94294d09fe59064f05a63 HTTP/1.1" 200 None 2026-05-05 02:17:49,165 - lava_callback - DEBUG - Processing node 69f94294d09fe59064f05a63: baseline-arm64-broonie 2026-05-05 02:17:49,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:17:49,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:17:49,365 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94294d09fe59064f05a63/log.txt.gz 2026-05-05 02:17:49,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:17:49,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:17:49,744 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94294d09fe59064f05a63/lava-logs.yaml 2026-05-05 02:17:49,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:17:49,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:17:49,855 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94294d09fe59064f05a63/lava_callback.json.gz 2026-05-05 02:17:49,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:17:49,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94294d09fe59064f05a63 HTTP/1.1" 200 None 2026-05-05 02:17:49,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:17:49,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 02:17:49,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:17:49,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94294d09fe59064f05a63 HTTP/1.1" 200 None 2026-05-05 02:17:49,981 - lava_callback - INFO - Completed processing callback for node 69f94294d09fe59064f05a63 INFO: 172.20.0.1:43400 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:18:04,127 - lava_callback - INFO - Received callback for node 69f94f29d09fe59064f063c9 INFO: 172.20.0.1:58214 - "POST /node/69f94f29d09fe59064f063c9 HTTP/1.0" 202 Accepted 2026-05-05 02:18:04,135 - lava_callback - INFO - Starting async job submit for node 69f94f29d09fe59064f063c9 2026-05-05 02:18:04,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:04,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94f29d09fe59064f063c9 HTTP/1.1" 200 None 2026-05-05 02:18:04,207 - lava_callback - DEBUG - Processing node 69f94f29d09fe59064f063c9: baseline-arm 2026-05-05 02:18:04,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:18:04,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:18:04,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69f94f29d09fe59064f063c9/log.txt.gz 2026-05-05 02:18:05,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:18:05,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:18:05,479 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69f94f29d09fe59064f063c9/lava-logs.yaml 2026-05-05 02:18:05,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:18:05,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:18:05,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69f94f29d09fe59064f063c9/lava_callback.json.gz DEBUG: Setup failed for 69f94f29d09fe59064f063c9. Transitting job node result: incomplete -> fail 2026-05-05 02:18:05,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:05,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94f29d09fe59064f063c9 HTTP/1.1" 200 None 2026-05-05 02:18:05,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:05,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 02:18:05,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:05,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94f29d09fe59064f063c9 HTTP/1.1" 200 None 2026-05-05 02:18:05,874 - lava_callback - INFO - Completed processing callback for node 69f94f29d09fe59064f063c9 INFO: 172.20.0.1:58228 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:18:14,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:14,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:18:27,587 - lava_callback - INFO - Received callback for node 69f6b06c1f903fec0b0df684 INFO: 172.20.0.1:36240 - "POST /node/69f6b06c1f903fec0b0df684 HTTP/1.0" 202 Accepted 2026-05-05 02:18:27,593 - lava_callback - INFO - Starting async job submit for node 69f6b06c1f903fec0b0df684 2026-05-05 02:18:27,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:27,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b06c1f903fec0b0df684 HTTP/1.1" 200 None 2026-05-05 02:18:27,608 - lava_callback - DEBUG - Processing node 69f6b06c1f903fec0b0df684: baseline-arm-clabbe 2026-05-05 02:18:27,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b06c1f903fec0b0df684 2026-05-05 02:18:27,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:27,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b06c1f903fec0b0df684 HTTP/1.1" 200 None 2026-05-05 02:18:27,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:27,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:18:27,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:18:27,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b06c1f903fec0b0df684 HTTP/1.1" 200 None 2026-05-05 02:18:27,660 - lava_callback - ERROR - Error processing callback for node 69f6b06c1f903fec0b0df684: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44936 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:21:07,681 - lava_callback - INFO - Received callback for node 69f6b0fd1f903fec0b0df6b5 INFO: 172.20.0.1:44950 - "POST /node/69f6b0fd1f903fec0b0df6b5 HTTP/1.0" 202 Accepted 2026-05-05 02:21:07,687 - lava_callback - INFO - Starting async job submit for node 69f6b0fd1f903fec0b0df6b5 2026-05-05 02:21:07,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:07,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b0fd1f903fec0b0df6b5 HTTP/1.1" 200 None 2026-05-05 02:21:07,707 - lava_callback - DEBUG - Processing node 69f6b0fd1f903fec0b0df6b5: baseline-arm-clabbe 2026-05-05 02:21:07,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b0fd1f903fec0b0df6b5 2026-05-05 02:21:07,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:07,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b0fd1f903fec0b0df6b5 HTTP/1.1" 200 None 2026-05-05 02:21:07,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:07,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:21:07,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:07,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b0fd1f903fec0b0df6b5 HTTP/1.1" 200 None 2026-05-05 02:21:07,761 - lava_callback - ERROR - Error processing callback for node 69f6b0fd1f903fec0b0df6b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60030 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:21:27,709 - lava_callback - INFO - Received callback for node 69f6b1221f903fec0b0df6c1 INFO: 172.20.0.1:60042 - "POST /node/69f6b1221f903fec0b0df6c1 HTTP/1.0" 202 Accepted 2026-05-05 02:21:27,717 - lava_callback - INFO - Starting async job submit for node 69f6b1221f903fec0b0df6c1 2026-05-05 02:21:27,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:27,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b1221f903fec0b0df6c1 HTTP/1.1" 200 None 2026-05-05 02:21:27,733 - lava_callback - DEBUG - Processing node 69f6b1221f903fec0b0df6c1: baseline-arm-clabbe 2026-05-05 02:21:27,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b1221f903fec0b0df6c1 2026-05-05 02:21:27,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:27,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b1221f903fec0b0df6c1 HTTP/1.1" 200 None 2026-05-05 02:21:27,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:27,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:21:27,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:21:27,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b1221f903fec0b0df6c1 HTTP/1.1" 200 None 2026-05-05 02:21:27,780 - lava_callback - ERROR - Error processing callback for node 69f6b1221f903fec0b0df6c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59110 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:22:07,715 - lava_callback - INFO - Received callback for node 69f6b1471f903fec0b0df6cf INFO: 172.20.0.1:59122 - "POST /node/69f6b1471f903fec0b0df6cf HTTP/1.0" 202 Accepted 2026-05-05 02:22:07,721 - lava_callback - INFO - Starting async job submit for node 69f6b1471f903fec0b0df6cf 2026-05-05 02:22:07,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:07,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b1471f903fec0b0df6cf HTTP/1.1" 200 None 2026-05-05 02:22:07,736 - lava_callback - DEBUG - Processing node 69f6b1471f903fec0b0df6cf: baseline-arm-clabbe 2026-05-05 02:22:07,736 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b1471f903fec0b0df6cf 2026-05-05 02:22:07,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:07,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b1471f903fec0b0df6cf HTTP/1.1" 200 None 2026-05-05 02:22:07,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:07,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:22:07,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:07,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b1471f903fec0b0df6cf HTTP/1.1" 200 None 2026-05-05 02:22:07,785 - lava_callback - ERROR - Error processing callback for node 69f6b1471f903fec0b0df6cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:22:27,378 - lava_callback - INFO - Received callback for node 69f94faad09fe59064f06414 INFO: 172.20.0.1:50642 - "POST /node/69f94faad09fe59064f06414 HTTP/1.0" 202 Accepted 2026-05-05 02:22:27,388 - lava_callback - INFO - Starting async job submit for node 69f94faad09fe59064f06414 2026-05-05 02:22:27,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:27,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94faad09fe59064f06414 HTTP/1.1" 200 None 2026-05-05 02:22:27,449 - lava_callback - DEBUG - Processing node 69f94faad09fe59064f06414: baseline-nfs-arm64 2026-05-05 02:22:28,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:22:28,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:22:28,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94faad09fe59064f06414/log.txt.gz 2026-05-05 02:22:29,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:22:29,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:22:29,600 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94faad09fe59064f06414/lava-logs.yaml 2026-05-05 02:22:29,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:58332 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:22:29,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:22:29,943 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f94faad09fe59064f06414/lava_callback.json.gz DEBUG: Setup failed for 69f94faad09fe59064f06414. Transitting job node result: incomplete -> fail 2026-05-05 02:22:30,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:30,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94faad09fe59064f06414 HTTP/1.1" 200 None 2026-05-05 02:22:30,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:30,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 02:22:30,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:30,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94faad09fe59064f06414 HTTP/1.1" 200 None 2026-05-05 02:22:30,185 - lava_callback - INFO - Completed processing callback for node 69f94faad09fe59064f06414 INFO: 172.20.0.1:58340 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:22:44,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:44,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45838 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:22:47,745 - lava_callback - INFO - Received callback for node 69f6b16b1f903fec0b0df6dc INFO: 172.20.0.1:45854 - "POST /node/69f6b16b1f903fec0b0df6dc HTTP/1.0" 202 Accepted 2026-05-05 02:22:47,751 - lava_callback - INFO - Starting async job submit for node 69f6b16b1f903fec0b0df6dc 2026-05-05 02:22:47,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:47,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b16b1f903fec0b0df6dc HTTP/1.1" 200 None 2026-05-05 02:22:47,768 - lava_callback - DEBUG - Processing node 69f6b16b1f903fec0b0df6dc: baseline-arm-clabbe 2026-05-05 02:22:47,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b16b1f903fec0b0df6dc 2026-05-05 02:22:47,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:47,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b16b1f903fec0b0df6dc HTTP/1.1" 200 None 2026-05-05 02:22:47,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:47,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:22:47,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:22:47,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b16b1f903fec0b0df6dc HTTP/1.1" 200 None 2026-05-05 02:22:47,815 - lava_callback - ERROR - Error processing callback for node 69f6b16b1f903fec0b0df6dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44592 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:23:27,785 - lava_callback - INFO - Received callback for node 69f6b18f1f903fec0b0df6e8 INFO: 172.20.0.1:44602 - "POST /node/69f6b18f1f903fec0b0df6e8 HTTP/1.0" 202 Accepted 2026-05-05 02:23:27,790 - lava_callback - INFO - Starting async job submit for node 69f6b18f1f903fec0b0df6e8 2026-05-05 02:23:27,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:23:27,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b18f1f903fec0b0df6e8 HTTP/1.1" 200 None 2026-05-05 02:23:27,813 - lava_callback - DEBUG - Processing node 69f6b18f1f903fec0b0df6e8: baseline-arm-clabbe 2026-05-05 02:23:27,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b18f1f903fec0b0df6e8 2026-05-05 02:23:27,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:23:27,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b18f1f903fec0b0df6e8 HTTP/1.1" 200 None 2026-05-05 02:23:27,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:23:27,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:23:27,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:23:27,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b18f1f903fec0b0df6e8 HTTP/1.1" 200 None 2026-05-05 02:23:27,861 - lava_callback - ERROR - Error processing callback for node 69f6b18f1f903fec0b0df6e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32914 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:24:07,786 - lava_callback - INFO - Received callback for node 69f6b1b41f903fec0b0df6f3 INFO: 172.20.0.1:32920 - "POST /node/69f6b1b41f903fec0b0df6f3 HTTP/1.0" 202 Accepted 2026-05-05 02:24:07,792 - lava_callback - INFO - Starting async job submit for node 69f6b1b41f903fec0b0df6f3 2026-05-05 02:24:07,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:24:07,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b1b41f903fec0b0df6f3 HTTP/1.1" 200 None 2026-05-05 02:24:07,807 - lava_callback - DEBUG - Processing node 69f6b1b41f903fec0b0df6f3: baseline-arm-clabbe 2026-05-05 02:24:07,808 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b1b41f903fec0b0df6f3 2026-05-05 02:24:07,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:24:07,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b1b41f903fec0b0df6f3 HTTP/1.1" 200 None 2026-05-05 02:24:07,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:24:07,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:24:07,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:24:07,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b1b41f903fec0b0df6f3 HTTP/1.1" 200 None 2026-05-05 02:24:07,853 - lava_callback - ERROR - Error processing callback for node 69f6b1b41f903fec0b0df6f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:27:07,969 - lava_callback - INFO - Received callback for node 69f6b26c1f903fec0b0df740 INFO: 172.20.0.1:48046 - "POST /node/69f6b26c1f903fec0b0df740 HTTP/1.0" 202 Accepted 2026-05-05 02:27:07,975 - lava_callback - INFO - Starting async job submit for node 69f6b26c1f903fec0b0df740 2026-05-05 02:27:07,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:27:07,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b26c1f903fec0b0df740 HTTP/1.1" 200 None 2026-05-05 02:27:07,991 - lava_callback - DEBUG - Processing node 69f6b26c1f903fec0b0df740: baseline-arm-clabbe 2026-05-05 02:27:07,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b26c1f903fec0b0df740 2026-05-05 02:27:07,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:27:08,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b26c1f903fec0b0df740 HTTP/1.1" 200 None 2026-05-05 02:27:08,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:27:08,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:27:08,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:27:08,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b26c1f903fec0b0df740 HTTP/1.1" 200 None 2026-05-05 02:27:08,041 - lava_callback - ERROR - Error processing callback for node 69f6b26c1f903fec0b0df740: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36334 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:28:08,296 - lava_callback - INFO - Received callback for node 69f95199d09fe59064f065ce INFO: 172.20.0.1:36344 - "POST /node/69f95199d09fe59064f065ce HTTP/1.0" 202 Accepted 2026-05-05 02:28:08,303 - lava_callback - INFO - Starting async job submit for node 69f95199d09fe59064f065ce 2026-05-05 02:28:08,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:28:08,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f95199d09fe59064f065ce HTTP/1.1" 200 None 2026-05-05 02:28:08,394 - lava_callback - DEBUG - Processing node 69f95199d09fe59064f065ce: ltp-smoke 2026-05-05 02:28:08,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:28:09,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:28:09,057 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f95199d09fe59064f065ce/log.txt.gz 2026-05-05 02:28:09,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:28:09,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:28:09,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f95199d09fe59064f065ce/lava-logs.yaml 2026-05-05 02:28:09,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:28:09,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:28:09,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f95199d09fe59064f065ce/lava_callback.json.gz 2026-05-05 02:28:09,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:28:09,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f95199d09fe59064f065ce HTTP/1.1" 200 None 2026-05-05 02:28:09,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:28:09,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93708d09fe59064f02cf1 HTTP/1.1" 200 None 2026-05-05 02:28:09,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:28:09,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f95199d09fe59064f065ce HTTP/1.1" 200 None 2026-05-05 02:28:09,964 - lava_callback - INFO - Completed processing callback for node 69f95199d09fe59064f065ce 2026-05-05 02:28:14,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:28:14,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53820 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:30:08,304 - lava_callback - INFO - Received callback for node 69f6b3251f903fec0b0df798 INFO: 172.20.0.1:53830 - "POST /node/69f6b3251f903fec0b0df798 HTTP/1.0" 202 Accepted 2026-05-05 02:30:08,309 - lava_callback - INFO - Starting async job submit for node 69f6b3251f903fec0b0df798 2026-05-05 02:30:08,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:30:08,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b3251f903fec0b0df798 HTTP/1.1" 200 None 2026-05-05 02:30:08,325 - lava_callback - DEBUG - Processing node 69f6b3251f903fec0b0df798: baseline-arm-clabbe 2026-05-05 02:30:08,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b3251f903fec0b0df798 2026-05-05 02:30:08,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:30:08,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b3251f903fec0b0df798 HTTP/1.1" 200 None 2026-05-05 02:30:08,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:30:08,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f343b9e61e72f6b29d1b08 HTTP/1.1" 200 None 2026-05-05 02:30:08,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:30:08,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b3251f903fec0b0df798 HTTP/1.1" 200 None 2026-05-05 02:30:08,391 - lava_callback - ERROR - Error processing callback for node 69f6b3251f903fec0b0df798: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38890 - "GET /node/69e6c58f60f5826562c157d3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41042 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:40:09,078 - lava_callback - INFO - Received callback for node 69f95366d09fe59064f06655 INFO: 172.20.0.1:41050 - "POST /node/69f95366d09fe59064f06655 HTTP/1.0" 202 Accepted 2026-05-05 02:40:09,084 - lava_callback - INFO - Starting async job submit for node 69f95366d09fe59064f06655 2026-05-05 02:40:09,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:40:09,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f95366d09fe59064f06655 HTTP/1.1" 200 None 2026-05-05 02:40:09,129 - lava_callback - DEBUG - Processing node 69f95366d09fe59064f06655: baseline-arm64-broonie 2026-05-05 02:40:09,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:40:09,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:40:09,509 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f95366d09fe59064f06655/log.txt.gz 2026-05-05 02:40:09,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:40:09,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:40:09,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f95366d09fe59064f06655/lava-logs.yaml 2026-05-05 02:40:09,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:40:10,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:40:10,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f95366d09fe59064f06655/lava_callback.json.gz 2026-05-05 02:40:10,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:40:10,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f95366d09fe59064f06655 HTTP/1.1" 200 None 2026-05-05 02:40:10,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:40:10,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 02:40:10,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:40:10,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f95366d09fe59064f06655 HTTP/1.1" 200 None 2026-05-05 02:40:10,156 - lava_callback - INFO - Completed processing callback for node 69f95366d09fe59064f06655 2026-05-05 02:40:14,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:40:14,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49792 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:49:32,255 - lava_callback - INFO - Received callback for node 69f6b7b31f903fec0b0df9ae INFO: 172.20.0.1:49796 - "POST /node/69f6b7b31f903fec0b0df9ae HTTP/1.0" 202 Accepted 2026-05-05 02:49:32,260 - lava_callback - INFO - Starting async job submit for node 69f6b7b31f903fec0b0df9ae 2026-05-05 02:49:32,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:49:32,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b7b31f903fec0b0df9ae HTTP/1.1" 200 None 2026-05-05 02:49:32,275 - lava_callback - DEBUG - Processing node 69f6b7b31f903fec0b0df9ae: baseline-arm-baylibre 2026-05-05 02:49:32,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b7b31f903fec0b0df9ae 2026-05-05 02:49:32,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:49:32,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b7b31f903fec0b0df9ae HTTP/1.1" 200 None 2026-05-05 02:49:32,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:49:32,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:49:32,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:49:32,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b7b31f903fec0b0df9ae HTTP/1.1" 200 None 2026-05-05 02:49:32,320 - lava_callback - ERROR - Error processing callback for node 69f6b7b31f903fec0b0df9ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50306 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:50:12,203 - lava_callback - INFO - Received callback for node 69f6b7d81f903fec0b0df9bb INFO: 172.20.0.1:35114 - "POST /node/69f6b7d81f903fec0b0df9bb HTTP/1.0" 202 Accepted 2026-05-05 02:50:12,208 - lava_callback - INFO - Starting async job submit for node 69f6b7d81f903fec0b0df9bb 2026-05-05 02:50:12,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:12,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b7d81f903fec0b0df9bb HTTP/1.1" 200 None 2026-05-05 02:50:12,223 - lava_callback - DEBUG - Processing node 69f6b7d81f903fec0b0df9bb: baseline-arm-baylibre 2026-05-05 02:50:12,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b7d81f903fec0b0df9bb 2026-05-05 02:50:12,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:12,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b7d81f903fec0b0df9bb HTTP/1.1" 200 None 2026-05-05 02:50:12,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:12,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:50:12,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:12,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b7d81f903fec0b0df9bb HTTP/1.1" 200 None 2026-05-05 02:50:12,270 - lava_callback - ERROR - Error processing callback for node 69f6b7d81f903fec0b0df9bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:50:52,453 - lava_callback - INFO - Received callback for node 69f6b8081f903fec0b0df9ce INFO: 172.20.0.1:50116 - "POST /node/69f6b8081f903fec0b0df9ce HTTP/1.0" 202 Accepted 2026-05-05 02:50:52,458 - lava_callback - INFO - Starting async job submit for node 69f6b8081f903fec0b0df9ce 2026-05-05 02:50:52,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:52,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b8081f903fec0b0df9ce HTTP/1.1" 200 None 2026-05-05 02:50:52,493 - lava_callback - DEBUG - Processing node 69f6b8081f903fec0b0df9ce: baseline-arm-baylibre 2026-05-05 02:50:52,493 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b8081f903fec0b0df9ce 2026-05-05 02:50:52,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:52,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b8081f903fec0b0df9ce HTTP/1.1" 200 None 2026-05-05 02:50:52,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:52,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:50:52,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:50:52,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b8081f903fec0b0df9ce HTTP/1.1" 200 None 2026-05-05 02:50:52,570 - lava_callback - ERROR - Error processing callback for node 69f6b8081f903fec0b0df9ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48236 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:51:32,286 - lava_callback - INFO - Received callback for node 69f6b82d1f903fec0b0df9de INFO: 172.20.0.1:48238 - "POST /node/69f6b82d1f903fec0b0df9de HTTP/1.0" 202 Accepted 2026-05-05 02:51:32,291 - lava_callback - INFO - Starting async job submit for node 69f6b82d1f903fec0b0df9de 2026-05-05 02:51:32,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:51:32,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b82d1f903fec0b0df9de HTTP/1.1" 200 None 2026-05-05 02:51:32,306 - lava_callback - DEBUG - Processing node 69f6b82d1f903fec0b0df9de: baseline-arm-baylibre 2026-05-05 02:51:32,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b82d1f903fec0b0df9de 2026-05-05 02:51:32,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:51:32,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b82d1f903fec0b0df9de HTTP/1.1" 200 None 2026-05-05 02:51:32,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:51:32,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:51:32,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:51:32,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b82d1f903fec0b0df9de HTTP/1.1" 200 None 2026-05-05 02:51:32,363 - lava_callback - ERROR - Error processing callback for node 69f6b82d1f903fec0b0df9de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48256 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:52:12,371 - lava_callback - INFO - Received callback for node 69f6b8541f903fec0b0df9fb INFO: 172.20.0.1:41252 - "POST /node/69f6b8541f903fec0b0df9fb HTTP/1.0" 202 Accepted 2026-05-05 02:52:12,377 - lava_callback - INFO - Starting async job submit for node 69f6b8541f903fec0b0df9fb 2026-05-05 02:52:12,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:12,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b8541f903fec0b0df9fb HTTP/1.1" 200 None 2026-05-05 02:52:12,393 - lava_callback - DEBUG - Processing node 69f6b8541f903fec0b0df9fb: baseline-arm-baylibre 2026-05-05 02:52:12,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b8541f903fec0b0df9fb 2026-05-05 02:52:12,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:12,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b8541f903fec0b0df9fb HTTP/1.1" 200 None 2026-05-05 02:52:12,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:12,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:52:12,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:12,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b8541f903fec0b0df9fb HTTP/1.1" 200 None 2026-05-05 02:52:12,437 - lava_callback - ERROR - Error processing callback for node 69f6b8541f903fec0b0df9fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44982 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:52:52,380 - lava_callback - INFO - Received callback for node 69f6b8791f903fec0b0dfa0d INFO: 172.20.0.1:47714 - "POST /node/69f6b8791f903fec0b0dfa0d HTTP/1.0" 202 Accepted 2026-05-05 02:52:52,385 - lava_callback - INFO - Starting async job submit for node 69f6b8791f903fec0b0dfa0d 2026-05-05 02:52:52,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:52,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b8791f903fec0b0dfa0d HTTP/1.1" 200 None 2026-05-05 02:52:52,401 - lava_callback - DEBUG - Processing node 69f6b8791f903fec0b0dfa0d: baseline-arm-baylibre 2026-05-05 02:52:52,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b8791f903fec0b0dfa0d 2026-05-05 02:52:52,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:52,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b8791f903fec0b0dfa0d HTTP/1.1" 200 None 2026-05-05 02:52:52,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:52,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:52:52,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:52:52,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b8791f903fec0b0dfa0d HTTP/1.1" 200 None 2026-05-05 02:52:52,450 - lava_callback - ERROR - Error processing callback for node 69f6b8791f903fec0b0dfa0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38822 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:55:49,324 - lava_callback - INFO - Received callback for node 69f6b9201f903fec0b0dfa68 INFO: 172.20.0.1:38826 - "POST /node/69f6b9201f903fec0b0dfa68 HTTP/1.0" 202 Accepted 2026-05-05 02:55:49,329 - lava_callback - INFO - Starting async job submit for node 69f6b9201f903fec0b0dfa68 2026-05-05 02:55:49,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:49,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b9201f903fec0b0dfa68 HTTP/1.1" 200 None 2026-05-05 02:55:49,388 - lava_callback - DEBUG - Processing node 69f6b9201f903fec0b0dfa68: baseline-arm-clabbe 2026-05-05 02:55:49,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b9201f903fec0b0dfa68 2026-05-05 02:55:49,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:49,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b9201f903fec0b0dfa68 HTTP/1.1" 200 None 2026-05-05 02:55:49,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:49,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:55:49,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:49,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b9201f903fec0b0dfa68 HTTP/1.1" 200 None 2026-05-05 02:55:49,486 - lava_callback - ERROR - Error processing callback for node 69f6b9201f903fec0b0dfa68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 02:55:49,586 - lava_callback - INFO - Received callback for node 69f955bbd09fe59064f066f6 INFO: 172.20.0.1:38832 - "POST /node/69f955bbd09fe59064f066f6 HTTP/1.0" 202 Accepted 2026-05-05 02:55:49,597 - lava_callback - INFO - Starting async job submit for node 69f955bbd09fe59064f066f6 2026-05-05 02:55:49,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:49,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f955bbd09fe59064f066f6 HTTP/1.1" 200 None 2026-05-05 02:55:49,680 - lava_callback - DEBUG - Processing node 69f955bbd09fe59064f066f6: ltp-smoke 2026-05-05 02:55:50,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:55:50,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:55:50,454 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f955bbd09fe59064f066f6/log.txt.gz 2026-05-05 02:55:50,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:55:50,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:55:50,972 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f955bbd09fe59064f066f6/lava-logs.yaml 2026-05-05 02:55:50,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 02:55:51,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 02:55:51,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f955bbd09fe59064f066f6/lava_callback.json.gz 2026-05-05 02:55:51,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:51,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f955bbd09fe59064f066f6 HTTP/1.1" 200 None 2026-05-05 02:55:51,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:51,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93708d09fe59064f02cf1 HTTP/1.1" 200 None 2026-05-05 02:55:51,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:55:51,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f955bbd09fe59064f066f6 HTTP/1.1" 200 None 2026-05-05 02:55:51,561 - lava_callback - INFO - Completed processing callback for node 69f955bbd09fe59064f066f6 INFO: 172.20.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37428 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:56:14,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:56:14,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50812 - "GET / HTTP/1.0" 200 OK 2026-05-05 02:59:09,316 - lava_callback - INFO - Received callback for node 69f6b9ed1f903fec0b0dfaad INFO: 172.20.0.1:50824 - "POST /node/69f6b9ed1f903fec0b0dfaad HTTP/1.0" 202 Accepted 2026-05-05 02:59:09,322 - lava_callback - INFO - Starting async job submit for node 69f6b9ed1f903fec0b0dfaad 2026-05-05 02:59:09,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:59:09,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b9ed1f903fec0b0dfaad HTTP/1.1" 200 None 2026-05-05 02:59:09,348 - lava_callback - DEBUG - Processing node 69f6b9ed1f903fec0b0dfaad: baseline-arm-clabbe 2026-05-05 02:59:09,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6b9ed1f903fec0b0dfaad 2026-05-05 02:59:09,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:59:09,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6b9ed1f903fec0b0dfaad HTTP/1.1" 200 None 2026-05-05 02:59:09,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:59:09,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 02:59:09,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 02:59:09,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6b9ed1f903fec0b0dfaad HTTP/1.1" 200 None 2026-05-05 02:59:09,419 - lava_callback - ERROR - Error processing callback for node 69f6b9ed1f903fec0b0dfaad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33544 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:00:49,310 - lava_callback - INFO - Received callback for node 69f6ba5c1f903fec0b0dfaeb INFO: 172.20.0.1:33558 - "POST /node/69f6ba5c1f903fec0b0dfaeb HTTP/1.0" 202 Accepted 2026-05-05 03:00:49,316 - lava_callback - INFO - Starting async job submit for node 69f6ba5c1f903fec0b0dfaeb 2026-05-05 03:00:49,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:00:49,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ba5c1f903fec0b0dfaeb HTTP/1.1" 200 None 2026-05-05 03:00:49,332 - lava_callback - DEBUG - Processing node 69f6ba5c1f903fec0b0dfaeb: baseline-arm-clabbe 2026-05-05 03:00:49,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ba5c1f903fec0b0dfaeb 2026-05-05 03:00:49,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:00:49,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ba5c1f903fec0b0dfaeb HTTP/1.1" 200 None 2026-05-05 03:00:49,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:00:49,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 03:00:49,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:00:49,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ba5c1f903fec0b0dfaeb HTTP/1.1" 200 None 2026-05-05 03:00:49,416 - lava_callback - ERROR - Error processing callback for node 69f6ba5c1f903fec0b0dfaeb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50266 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:01:29,321 - lava_callback - INFO - Received callback for node 69f6ba801f903fec0b0dfb04 INFO: 172.20.0.1:50274 - "POST /node/69f6ba801f903fec0b0dfb04 HTTP/1.0" 202 Accepted 2026-05-05 03:01:29,327 - lava_callback - INFO - Starting async job submit for node 69f6ba801f903fec0b0dfb04 2026-05-05 03:01:29,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:01:29,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ba801f903fec0b0dfb04 HTTP/1.1" 200 None 2026-05-05 03:01:29,342 - lava_callback - DEBUG - Processing node 69f6ba801f903fec0b0dfb04: baseline-arm-clabbe 2026-05-05 03:01:29,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ba801f903fec0b0dfb04 2026-05-05 03:01:29,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:01:29,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ba801f903fec0b0dfb04 HTTP/1.1" 200 None 2026-05-05 03:01:29,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:01:29,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 03:01:29,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:01:29,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ba801f903fec0b0dfb04 HTTP/1.1" 200 None 2026-05-05 03:01:29,388 - lava_callback - ERROR - Error processing callback for node 69f6ba801f903fec0b0dfb04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:05:10,578 - lava_callback - INFO - Received callback for node 69f6bb561f903fec0b0dfb44 INFO: 172.20.0.1:35774 - "POST /node/69f6bb561f903fec0b0dfb44 HTTP/1.0" 202 Accepted 2026-05-05 03:05:10,584 - lava_callback - INFO - Starting async job submit for node 69f6bb561f903fec0b0dfb44 2026-05-05 03:05:10,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:05:10,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bb561f903fec0b0dfb44 HTTP/1.1" 200 None 2026-05-05 03:05:10,599 - lava_callback - DEBUG - Processing node 69f6bb561f903fec0b0dfb44: baseline-arm-clabbe 2026-05-05 03:05:10,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6bb561f903fec0b0dfb44 2026-05-05 03:05:10,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:05:10,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bb561f903fec0b0dfb44 HTTP/1.1" 200 None 2026-05-05 03:05:10,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:05:10,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b3718cb8354ccaa5e9d2 HTTP/1.1" 200 None 2026-05-05 03:05:10,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:05:10,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6bb561f903fec0b0dfb44 HTTP/1.1" 200 None 2026-05-05 03:05:10,658 - lava_callback - ERROR - Error processing callback for node 69f6bb561f903fec0b0dfb44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48520 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:13:30,853 - lava_callback - INFO - Received callback for node 69f95c39d09fe59064f0672d INFO: 172.20.0.1:48530 - "POST /node/69f95c39d09fe59064f0672d HTTP/1.0" 202 Accepted 2026-05-05 03:13:30,860 - lava_callback - INFO - Starting async job submit for node 69f95c39d09fe59064f0672d 2026-05-05 03:13:30,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:13:30,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f95c39d09fe59064f0672d HTTP/1.1" 200 None 2026-05-05 03:13:30,947 - lava_callback - DEBUG - Processing node 69f95c39d09fe59064f0672d: ltp-smoke 2026-05-05 03:13:31,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 03:13:31,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 03:13:31,729 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f95c39d09fe59064f0672d/log.txt.gz 2026-05-05 03:13:32,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 03:13:32,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 03:13:32,204 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f95c39d09fe59064f0672d/lava-logs.yaml 2026-05-05 03:13:32,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 03:13:32,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 03:13:32,356 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f95c39d09fe59064f0672d/lava_callback.json.gz 2026-05-05 03:13:32,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:13:32,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f95c39d09fe59064f0672d HTTP/1.1" 200 None 2026-05-05 03:13:32,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:13:32,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93708d09fe59064f02cf1 HTTP/1.1" 200 None 2026-05-05 03:13:32,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:13:32,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f95c39d09fe59064f0672d HTTP/1.1" 200 None 2026-05-05 03:13:32,629 - lava_callback - INFO - Completed processing callback for node 69f95c39d09fe59064f0672d INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:13:44,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:13:44,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53754 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:18:13,860 - lava_callback - INFO - Received callback for node 69f6be671f903fec0b0dfc1c INFO: 172.20.0.1:55140 - "POST /node/69f6be671f903fec0b0dfc1c HTTP/1.0" 202 Accepted 2026-05-05 03:18:13,866 - lava_callback - INFO - Starting async job submit for node 69f6be671f903fec0b0dfc1c 2026-05-05 03:18:13,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:18:13,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6be671f903fec0b0dfc1c HTTP/1.1" 200 None 2026-05-05 03:18:13,893 - lava_callback - DEBUG - Processing node 69f6be671f903fec0b0dfc1c: baseline-arm64-baylibre 2026-05-05 03:18:13,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6be671f903fec0b0dfc1c 2026-05-05 03:18:13,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:18:13,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6be671f903fec0b0dfc1c HTTP/1.1" 200 None 2026-05-05 03:18:13,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:18:13,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:18:13,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:18:13,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6be671f903fec0b0dfc1c HTTP/1.1" 200 None 2026-05-05 03:18:13,940 - lava_callback - ERROR - Error processing callback for node 69f6be671f903fec0b0dfc1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54014 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:19:13,752 - lava_callback - INFO - Received callback for node 69f6beb01f903fec0b0dfc44 INFO: 172.20.0.1:36918 - "POST /node/69f6beb01f903fec0b0dfc44 HTTP/1.0" 202 Accepted 2026-05-05 03:19:13,758 - lava_callback - INFO - Starting async job submit for node 69f6beb01f903fec0b0dfc44 2026-05-05 03:19:13,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:19:13,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6beb01f903fec0b0dfc44 HTTP/1.1" 200 None 2026-05-05 03:19:13,773 - lava_callback - DEBUG - Processing node 69f6beb01f903fec0b0dfc44: baseline-arm64-baylibre 2026-05-05 03:19:13,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6beb01f903fec0b0dfc44 2026-05-05 03:19:13,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:19:13,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6beb01f903fec0b0dfc44 HTTP/1.1" 200 None 2026-05-05 03:19:13,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:19:13,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:19:13,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:19:13,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6beb01f903fec0b0dfc44 HTTP/1.1" 200 None 2026-05-05 03:19:13,820 - lava_callback - ERROR - Error processing callback for node 69f6beb01f903fec0b0dfc44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59598 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:20:33,944 - lava_callback - INFO - Received callback for node 69f6bef91f903fec0b0dfc6b INFO: 172.20.0.1:59610 - "POST /node/69f6bef91f903fec0b0dfc6b HTTP/1.0" 202 Accepted 2026-05-05 03:20:33,949 - lava_callback - INFO - Starting async job submit for node 69f6bef91f903fec0b0dfc6b 2026-05-05 03:20:33,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:20:33,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bef91f903fec0b0dfc6b HTTP/1.1" 200 None 2026-05-05 03:20:33,968 - lava_callback - DEBUG - Processing node 69f6bef91f903fec0b0dfc6b: baseline-arm64-baylibre 2026-05-05 03:20:33,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6bef91f903fec0b0dfc6b 2026-05-05 03:20:33,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:20:33,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bef91f903fec0b0dfc6b HTTP/1.1" 200 None 2026-05-05 03:20:33,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:20:33,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:20:33,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:20:34,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6bef91f903fec0b0dfc6b HTTP/1.1" 200 None 2026-05-05 03:20:34,013 - lava_callback - ERROR - Error processing callback for node 69f6bef91f903fec0b0dfc6b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:22:13,972 - lava_callback - INFO - Received callback for node 69f6bf591f903fec0b0dfc9b INFO: 172.20.0.1:57602 - "POST /node/69f6bf591f903fec0b0dfc9b HTTP/1.0" 202 Accepted 2026-05-05 03:22:13,977 - lava_callback - INFO - Starting async job submit for node 69f6bf591f903fec0b0dfc9b 2026-05-05 03:22:13,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:22:13,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bf591f903fec0b0dfc9b HTTP/1.1" 200 None 2026-05-05 03:22:13,993 - lava_callback - DEBUG - Processing node 69f6bf591f903fec0b0dfc9b: baseline-arm64-baylibre 2026-05-05 03:22:13,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6bf591f903fec0b0dfc9b 2026-05-05 03:22:13,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:22:14,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bf591f903fec0b0dfc9b HTTP/1.1" 200 None 2026-05-05 03:22:14,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:22:14,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:22:14,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:22:14,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6bf591f903fec0b0dfc9b HTTP/1.1" 200 None 2026-05-05 03:22:14,052 - lava_callback - ERROR - Error processing callback for node 69f6bf591f903fec0b0dfc9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60612 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:23:34,042 - lava_callback - INFO - Received callback for node 69f6bfa31f903fec0b0dfcc6 INFO: 172.20.0.1:60620 - "POST /node/69f6bfa31f903fec0b0dfcc6 HTTP/1.0" 202 Accepted 2026-05-05 03:23:34,047 - lava_callback - INFO - Starting async job submit for node 69f6bfa31f903fec0b0dfcc6 2026-05-05 03:23:34,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:23:34,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bfa31f903fec0b0dfcc6 HTTP/1.1" 200 None 2026-05-05 03:23:34,063 - lava_callback - DEBUG - Processing node 69f6bfa31f903fec0b0dfcc6: baseline-arm64-baylibre 2026-05-05 03:23:34,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6bfa31f903fec0b0dfcc6 2026-05-05 03:23:34,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:23:34,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bfa31f903fec0b0dfcc6 HTTP/1.1" 200 None 2026-05-05 03:23:34,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:23:34,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:23:34,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:23:34,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6bfa31f903fec0b0dfcc6 HTTP/1.1" 200 None 2026-05-05 03:23:34,155 - lava_callback - ERROR - Error processing callback for node 69f6bfa31f903fec0b0dfcc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56992 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:24:34,057 - lava_callback - INFO - Received callback for node 69f6bfed1f903fec0b0dfceb INFO: 172.20.0.1:56998 - "POST /node/69f6bfed1f903fec0b0dfceb HTTP/1.0" 202 Accepted 2026-05-05 03:24:34,062 - lava_callback - INFO - Starting async job submit for node 69f6bfed1f903fec0b0dfceb 2026-05-05 03:24:34,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:24:34,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bfed1f903fec0b0dfceb HTTP/1.1" 200 None 2026-05-05 03:24:34,078 - lava_callback - DEBUG - Processing node 69f6bfed1f903fec0b0dfceb: baseline-arm64-baylibre 2026-05-05 03:24:34,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6bfed1f903fec0b0dfceb 2026-05-05 03:24:34,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:24:34,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6bfed1f903fec0b0dfceb HTTP/1.1" 200 None 2026-05-05 03:24:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:24:34,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:24:34,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:24:34,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6bfed1f903fec0b0dfceb HTTP/1.1" 200 None 2026-05-05 03:24:34,123 - lava_callback - ERROR - Error processing callback for node 69f6bfed1f903fec0b0dfceb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47312 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:26:30,323 - lava_callback - INFO - Received callback for node 69f6c0501f903fec0b0dfd2a INFO: 172.20.0.1:38600 - "POST /node/69f6c0501f903fec0b0dfd2a HTTP/1.0" 202 Accepted 2026-05-05 03:26:30,329 - lava_callback - INFO - Starting async job submit for node 69f6c0501f903fec0b0dfd2a 2026-05-05 03:26:30,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:26:30,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c0501f903fec0b0dfd2a HTTP/1.1" 200 None 2026-05-05 03:26:30,343 - lava_callback - DEBUG - Processing node 69f6c0501f903fec0b0dfd2a: baseline-arm64-clabbe 2026-05-05 03:26:30,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6c0501f903fec0b0dfd2a 2026-05-05 03:26:30,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:26:30,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c0501f903fec0b0dfd2a HTTP/1.1" 200 None 2026-05-05 03:26:30,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:26:30,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:26:30,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:26:30,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6c0501f903fec0b0dfd2a HTTP/1.1" 200 None 2026-05-05 03:26:30,389 - lava_callback - ERROR - Error processing callback for node 69f6c0501f903fec0b0dfd2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49972 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:28:30,601 - lava_callback - INFO - Received callback for node 69f6c0c81f903fec0b0dfd55 INFO: 172.20.0.1:35054 - "POST /node/69f6c0c81f903fec0b0dfd55 HTTP/1.0" 202 Accepted 2026-05-05 03:28:30,606 - lava_callback - INFO - Starting async job submit for node 69f6c0c81f903fec0b0dfd55 2026-05-05 03:28:30,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:28:30,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c0c81f903fec0b0dfd55 HTTP/1.1" 200 None 2026-05-05 03:28:30,622 - lava_callback - DEBUG - Processing node 69f6c0c81f903fec0b0dfd55: baseline-arm64-clabbe 2026-05-05 03:28:30,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6c0c81f903fec0b0dfd55 2026-05-05 03:28:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:28:30,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c0c81f903fec0b0dfd55 HTTP/1.1" 200 None 2026-05-05 03:28:30,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:28:30,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:28:30,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:28:30,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6c0c81f903fec0b0dfd55 HTTP/1.1" 200 None 2026-05-05 03:28:30,681 - lava_callback - ERROR - Error processing callback for node 69f6c0c81f903fec0b0dfd55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:29:30,396 - lava_callback - INFO - Received callback for node 69f6c1131f903fec0b0dfd77 INFO: 172.20.0.1:47144 - "POST /node/69f6c1131f903fec0b0dfd77 HTTP/1.0" 202 Accepted 2026-05-05 03:29:30,401 - lava_callback - INFO - Starting async job submit for node 69f6c1131f903fec0b0dfd77 2026-05-05 03:29:30,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:29:30,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c1131f903fec0b0dfd77 HTTP/1.1" 200 None 2026-05-05 03:29:30,417 - lava_callback - DEBUG - Processing node 69f6c1131f903fec0b0dfd77: baseline-arm64-clabbe 2026-05-05 03:29:30,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6c1131f903fec0b0dfd77 2026-05-05 03:29:30,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:29:30,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c1131f903fec0b0dfd77 HTTP/1.1" 200 None 2026-05-05 03:29:30,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:29:30,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:29:30,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:29:30,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6c1131f903fec0b0dfd77 HTTP/1.1" 200 None 2026-05-05 03:29:30,465 - lava_callback - ERROR - Error processing callback for node 69f6c1131f903fec0b0dfd77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:30:50,535 - lava_callback - INFO - Received callback for node 69f6c15c1f903fec0b0dfdb0 INFO: 172.20.0.1:42514 - "POST /node/69f6c15c1f903fec0b0dfdb0 HTTP/1.0" 202 Accepted 2026-05-05 03:30:50,540 - lava_callback - INFO - Starting async job submit for node 69f6c15c1f903fec0b0dfdb0 2026-05-05 03:30:50,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:30:50,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c15c1f903fec0b0dfdb0 HTTP/1.1" 200 None 2026-05-05 03:30:50,568 - lava_callback - DEBUG - Processing node 69f6c15c1f903fec0b0dfdb0: baseline-arm64-clabbe 2026-05-05 03:30:50,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6c15c1f903fec0b0dfdb0 2026-05-05 03:30:50,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:30:50,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c15c1f903fec0b0dfdb0 HTTP/1.1" 200 None 2026-05-05 03:30:50,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:30:50,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:30:50,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:30:50,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6c15c1f903fec0b0dfdb0 HTTP/1.1" 200 None 2026-05-05 03:30:50,617 - lava_callback - ERROR - Error processing callback for node 69f6c15c1f903fec0b0dfdb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33676 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:33:10,585 - lava_callback - INFO - Received callback for node 69f6c1ef1f903fec0b0dfe09 INFO: 172.20.0.1:47704 - "POST /node/69f6c1ef1f903fec0b0dfe09 HTTP/1.0" 202 Accepted 2026-05-05 03:33:10,591 - lava_callback - INFO - Starting async job submit for node 69f6c1ef1f903fec0b0dfe09 2026-05-05 03:33:10,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:33:10,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c1ef1f903fec0b0dfe09 HTTP/1.1" 200 None 2026-05-05 03:33:10,606 - lava_callback - DEBUG - Processing node 69f6c1ef1f903fec0b0dfe09: baseline-arm64-clabbe 2026-05-05 03:33:10,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6c1ef1f903fec0b0dfe09 2026-05-05 03:33:10,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:33:10,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c1ef1f903fec0b0dfe09 HTTP/1.1" 200 None 2026-05-05 03:33:10,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:33:10,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:33:10,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:33:10,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6c1ef1f903fec0b0dfe09 HTTP/1.1" 200 None 2026-05-05 03:33:10,661 - lava_callback - ERROR - Error processing callback for node 69f6c1ef1f903fec0b0dfe09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:34:30,619 - lava_callback - INFO - Received callback for node 69f6c2391f903fec0b0dfe29 INFO: 172.20.0.1:43450 - "POST /node/69f6c2391f903fec0b0dfe29 HTTP/1.0" 202 Accepted 2026-05-05 03:34:30,625 - lava_callback - INFO - Starting async job submit for node 69f6c2391f903fec0b0dfe29 2026-05-05 03:34:30,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:30,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c2391f903fec0b0dfe29 HTTP/1.1" 200 None 2026-05-05 03:34:30,644 - lava_callback - DEBUG - Processing node 69f6c2391f903fec0b0dfe29: baseline-arm64-clabbe 2026-05-05 03:34:30,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6c2391f903fec0b0dfe29 2026-05-05 03:34:30,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:30,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6c2391f903fec0b0dfe29 HTTP/1.1" 200 None 2026-05-05 03:34:30,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:30,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f5b37d8cb8354ccaa5e9e1 HTTP/1.1" 200 None 2026-05-05 03:34:30,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:30,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6c2391f903fec0b0dfe29 HTTP/1.1" 200 None 2026-05-05 03:34:30,695 - lava_callback - ERROR - Error processing callback for node 69f6c2391f903fec0b0dfe29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43464 - "GET / HTTP/1.0" 200 OK 2026-05-05 03:34:37,860 - lava_callback - INFO - Received callback for node 69f9546fd09fe59064f066b0 INFO: 172.20.0.1:43478 - "POST /node/69f9546fd09fe59064f066b0 HTTP/1.0" 202 Accepted 2026-05-05 03:34:37,868 - lava_callback - INFO - Starting async job submit for node 69f9546fd09fe59064f066b0 2026-05-05 03:34:37,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:37,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9546fd09fe59064f066b0 HTTP/1.1" 200 None 2026-05-05 03:34:37,926 - lava_callback - DEBUG - Processing node 69f9546fd09fe59064f066b0: baseline-nfs-arm64 2026-05-05 03:34:39,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 03:34:39,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 03:34:39,317 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9546fd09fe59064f066b0/log.txt.gz 2026-05-05 03:34:39,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 03:34:40,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 03:34:40,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9546fd09fe59064f066b0/lava-logs.yaml 2026-05-05 03:34:40,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 03:34:40,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 03:34:40,368 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69f9546fd09fe59064f066b0/lava_callback.json.gz DEBUG: Setup failed for 69f9546fd09fe59064f066b0. Transitting job node result: incomplete -> fail 2026-05-05 03:34:40,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:40,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9546fd09fe59064f066b0 HTTP/1.1" 200 None 2026-05-05 03:34:40,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:40,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 03:34:40,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:40,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9546fd09fe59064f066b0 HTTP/1.1" 200 None 2026-05-05 03:34:40,609 - lava_callback - INFO - Completed processing callback for node 69f9546fd09fe59064f066b0 2026-05-05 03:34:44,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 03:34:44,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32942 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:00:40,178 - lava_callback - INFO - Received callback for node 69f9428ed09fe59064f05a54 INFO: 172.20.0.1:43796 - "POST /node/69f9428ed09fe59064f05a54 HTTP/1.0" 202 Accepted 2026-05-05 04:00:40,184 - lava_callback - INFO - Starting async job submit for node 69f9428ed09fe59064f05a54 2026-05-05 04:00:40,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:00:40,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9428ed09fe59064f05a54 HTTP/1.1" 200 None 2026-05-05 04:00:40,224 - lava_callback - DEBUG - Processing node 69f9428ed09fe59064f05a54: baseline-arm64-broonie 2026-05-05 04:00:40,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:00:40,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:00:40,755 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428ed09fe59064f05a54/log.txt.gz 2026-05-05 04:00:40,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:00:41,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:00:41,099 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428ed09fe59064f05a54/lava-logs.yaml 2026-05-05 04:00:41,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:00:41,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:00:41,397 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428ed09fe59064f05a54/lava_callback.json.gz 2026-05-05 04:00:41,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:00:41,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9428ed09fe59064f05a54 HTTP/1.1" 200 None 2026-05-05 04:00:41,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:00:41,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 04:00:41,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:00:41,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9428ed09fe59064f05a54 HTTP/1.1" 200 None 2026-05-05 04:00:41,494 - lava_callback - INFO - Completed processing callback for node 69f9428ed09fe59064f05a54 2026-05-05 04:00:44,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:00:44,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46404 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:45:52,962 - lava_callback - INFO - Received callback for node 69f94291d09fe59064f05a5c INFO: 172.20.0.1:45494 - "POST /node/69f94291d09fe59064f05a5c HTTP/1.0" 202 Accepted 2026-05-05 04:45:52,970 - lava_callback - INFO - Starting async job submit for node 69f94291d09fe59064f05a5c 2026-05-05 04:45:52,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:45:53,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94291d09fe59064f05a5c HTTP/1.1" 200 None 2026-05-05 04:45:53,011 - lava_callback - DEBUG - Processing node 69f94291d09fe59064f05a5c: baseline-arm64-broonie 2026-05-05 04:45:53,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:45:53,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:45:53,538 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94291d09fe59064f05a5c/log.txt.gz 2026-05-05 04:45:53,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:45:53,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:45:53,866 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94291d09fe59064f05a5c/lava-logs.yaml 2026-05-05 04:45:53,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:45:54,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:45:54,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94291d09fe59064f05a5c/lava_callback.json.gz 2026-05-05 04:45:54,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:45:54,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94291d09fe59064f05a5c HTTP/1.1" 200 None 2026-05-05 04:45:54,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:45:54,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 04:45:54,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:45:54,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94291d09fe59064f05a5c HTTP/1.1" 200 None 2026-05-05 04:45:54,087 - lava_callback - INFO - Completed processing callback for node 69f94291d09fe59064f05a5c INFO: 172.20.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43122 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:46:14,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:46:14,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:49:33,581 - lava_callback - INFO - Received callback for node 69f6d3ca1f903fec0b0dff37 INFO: 172.20.0.1:53526 - "POST /node/69f6d3ca1f903fec0b0dff37 HTTP/1.0" 202 Accepted 2026-05-05 04:49:33,587 - lava_callback - INFO - Starting async job submit for node 69f6d3ca1f903fec0b0dff37 2026-05-05 04:49:33,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:49:33,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d3ca1f903fec0b0dff37 HTTP/1.1" 200 None 2026-05-05 04:49:33,602 - lava_callback - DEBUG - Processing node 69f6d3ca1f903fec0b0dff37: baseline-arm-clabbe 2026-05-05 04:49:33,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d3ca1f903fec0b0dff37 2026-05-05 04:49:33,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:49:33,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d3ca1f903fec0b0dff37 HTTP/1.1" 200 None 2026-05-05 04:49:33,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:49:33,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623a28234569b79a1c04a HTTP/1.1" 200 None 2026-05-05 04:49:33,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:49:33,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d3ca1f903fec0b0dff37 HTTP/1.1" 200 None 2026-05-05 04:49:33,647 - lava_callback - ERROR - Error processing callback for node 69f6d3ca1f903fec0b0dff37: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55824 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:51:33,637 - lava_callback - INFO - Received callback for node 69f6d4471f903fec0b0dff6e INFO: 172.20.0.1:55832 - "POST /node/69f6d4471f903fec0b0dff6e HTTP/1.0" 202 Accepted 2026-05-05 04:51:33,642 - lava_callback - INFO - Starting async job submit for node 69f6d4471f903fec0b0dff6e 2026-05-05 04:51:33,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:51:33,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d4471f903fec0b0dff6e HTTP/1.1" 200 None 2026-05-05 04:51:33,658 - lava_callback - DEBUG - Processing node 69f6d4471f903fec0b0dff6e: baseline-arm-clabbe 2026-05-05 04:51:33,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d4471f903fec0b0dff6e 2026-05-05 04:51:33,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:51:33,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d4471f903fec0b0dff6e HTTP/1.1" 200 None 2026-05-05 04:51:33,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:51:33,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623a28234569b79a1c04a HTTP/1.1" 200 None 2026-05-05 04:51:33,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:51:33,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d4471f903fec0b0dff6e HTTP/1.1" 200 None 2026-05-05 04:51:33,704 - lava_callback - ERROR - Error processing callback for node 69f6d4471f903fec0b0dff6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38010 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:52:13,637 - lava_callback - INFO - Received callback for node 69f6d46d1f903fec0b0dff7d INFO: 172.20.0.1:36946 - "POST /node/69f6d46d1f903fec0b0dff7d HTTP/1.0" 202 Accepted 2026-05-05 04:52:13,643 - lava_callback - INFO - Starting async job submit for node 69f6d46d1f903fec0b0dff7d 2026-05-05 04:52:13,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:13,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d46d1f903fec0b0dff7d HTTP/1.1" 200 None 2026-05-05 04:52:13,658 - lava_callback - DEBUG - Processing node 69f6d46d1f903fec0b0dff7d: baseline-arm-clabbe 2026-05-05 04:52:13,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d46d1f903fec0b0dff7d 2026-05-05 04:52:13,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:13,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d46d1f903fec0b0dff7d HTTP/1.1" 200 None 2026-05-05 04:52:13,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:13,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623a28234569b79a1c04a HTTP/1.1" 200 None 2026-05-05 04:52:13,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:13,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d46d1f903fec0b0dff7d HTTP/1.1" 200 None 2026-05-05 04:52:13,705 - lava_callback - ERROR - Error processing callback for node 69f6d46d1f903fec0b0dff7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:52:53,665 - lava_callback - INFO - Received callback for node 69f6d4921f903fec0b0dff8f INFO: 172.20.0.1:40680 - "POST /node/69f6d4921f903fec0b0dff8f HTTP/1.0" 202 Accepted 2026-05-05 04:52:53,671 - lava_callback - INFO - Starting async job submit for node 69f6d4921f903fec0b0dff8f 2026-05-05 04:52:53,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:53,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d4921f903fec0b0dff8f HTTP/1.1" 200 None 2026-05-05 04:52:53,688 - lava_callback - DEBUG - Processing node 69f6d4921f903fec0b0dff8f: baseline-arm-clabbe 2026-05-05 04:52:53,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d4921f903fec0b0dff8f 2026-05-05 04:52:53,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:53,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d4921f903fec0b0dff8f HTTP/1.1" 200 None 2026-05-05 04:52:53,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:53,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623a28234569b79a1c04a HTTP/1.1" 200 None 2026-05-05 04:52:53,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:52:53,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d4921f903fec0b0dff8f HTTP/1.1" 200 None 2026-05-05 04:52:53,748 - lava_callback - ERROR - Error processing callback for node 69f6d4921f903fec0b0dff8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54542 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:54:33,802 - lava_callback - INFO - Received callback for node 69f6d5011f903fec0b0dffba INFO: 172.20.0.1:54556 - "POST /node/69f6d5011f903fec0b0dffba HTTP/1.0" 202 Accepted 2026-05-05 04:54:33,807 - lava_callback - INFO - Starting async job submit for node 69f6d5011f903fec0b0dffba 2026-05-05 04:54:33,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:54:33,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d5011f903fec0b0dffba HTTP/1.1" 200 None 2026-05-05 04:54:33,823 - lava_callback - DEBUG - Processing node 69f6d5011f903fec0b0dffba: baseline-arm-clabbe 2026-05-05 04:54:33,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d5011f903fec0b0dffba 2026-05-05 04:54:33,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:54:33,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d5011f903fec0b0dffba HTTP/1.1" 200 None 2026-05-05 04:54:33,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:54:33,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623a28234569b79a1c04a HTTP/1.1" 200 None 2026-05-05 04:54:33,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:54:33,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d5011f903fec0b0dffba HTTP/1.1" 200 None 2026-05-05 04:54:33,875 - lava_callback - ERROR - Error processing callback for node 69f6d5011f903fec0b0dffba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45138 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:56:21,485 - lava_callback - INFO - Received callback for node 69f93ca0d09fe59064f055ea INFO: 172.20.0.1:38364 - "POST /node/69f93ca0d09fe59064f055ea HTTP/1.0" 202 Accepted 2026-05-05 04:56:21,491 - lava_callback - INFO - Starting async job submit for node 69f93ca0d09fe59064f055ea 2026-05-05 04:56:21,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:56:21,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca0d09fe59064f055ea HTTP/1.1" 200 None 2026-05-05 04:56:21,530 - lava_callback - DEBUG - Processing node 69f93ca0d09fe59064f055ea: ltp-cve 2026-05-05 04:56:21,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:56:22,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:56:22,060 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69f93ca0d09fe59064f055ea/log.txt.gz 2026-05-05 04:56:22,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:56:22,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:56:22,619 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69f93ca0d09fe59064f055ea/lava-logs.yaml 2026-05-05 04:56:22,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:56:22,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:56:22,789 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69f93ca0d09fe59064f055ea/lava_callback.json.gz 2026-05-05 04:56:22,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:56:22,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca0d09fe59064f055ea HTTP/1.1" 200 None 2026-05-05 04:56:22,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:56:22,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 04:56:22,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:56:22,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ca0d09fe59064f055ea HTTP/1.1" 200 None 2026-05-05 04:56:22,866 - lava_callback - INFO - Completed processing callback for node 69f93ca0d09fe59064f055ea INFO: 172.20.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38590 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:56:44,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:56:44,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58576 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:57:41,541 - lava_callback - INFO - Received callback for node 69f93ca3d09fe59064f055f4 INFO: 172.20.0.1:32796 - "POST /node/69f93ca3d09fe59064f055f4 HTTP/1.0" 202 Accepted 2026-05-05 04:57:41,549 - lava_callback - INFO - Starting async job submit for node 69f93ca3d09fe59064f055f4 2026-05-05 04:57:41,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:57:41,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca3d09fe59064f055f4 HTTP/1.1" 200 None 2026-05-05 04:57:41,607 - lava_callback - DEBUG - Processing node 69f93ca3d09fe59064f055f4: ltp-fs_bind 2026-05-05 04:57:42,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:57:42,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:57:42,141 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69f93ca3d09fe59064f055f4/log.txt.gz 2026-05-05 04:57:42,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:57:42,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:57:42,540 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69f93ca3d09fe59064f055f4/lava-logs.yaml 2026-05-05 04:57:42,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:57:42,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:57:42,663 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69f93ca3d09fe59064f055f4/lava_callback.json.gz 2026-05-05 04:57:42,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:57:42,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca3d09fe59064f055f4 HTTP/1.1" 200 None 2026-05-05 04:57:42,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:57:42,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 04:57:42,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:57:42,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ca3d09fe59064f055f4 HTTP/1.1" 200 None 2026-05-05 04:57:42,745 - lava_callback - INFO - Completed processing callback for node 69f93ca3d09fe59064f055f4 2026-05-05 04:57:44,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:57:44,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:58:13,588 - lava_callback - INFO - Received callback for node 69f93ca5d09fe59064f055f7 INFO: 172.20.0.1:47462 - "POST /node/69f93ca5d09fe59064f055f7 HTTP/1.0" 202 Accepted 2026-05-05 04:58:13,594 - lava_callback - INFO - Starting async job submit for node 69f93ca5d09fe59064f055f7 2026-05-05 04:58:13,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:13,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca5d09fe59064f055f7 HTTP/1.1" 200 None 2026-05-05 04:58:13,636 - lava_callback - DEBUG - Processing node 69f93ca5d09fe59064f055f7: ltp-fsx 2026-05-05 04:58:13,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:58:14,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:58:14,131 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69f93ca5d09fe59064f055f7/log.txt.gz 2026-05-05 04:58:14,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:58:14,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:58:14,711 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69f93ca5d09fe59064f055f7/lava-logs.yaml 2026-05-05 04:58:14,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:58:15,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:58:15,019 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69f93ca5d09fe59064f055f7/lava_callback.json.gz 2026-05-05 04:58:15,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:15,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca5d09fe59064f055f7 HTTP/1.1" 200 None 2026-05-05 04:58:15,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:15,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 04:58:15,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ca5d09fe59064f055f7 HTTP/1.1" 200 None 2026-05-05 04:58:15,098 - lava_callback - INFO - Completed processing callback for node 69f93ca5d09fe59064f055f7 INFO: 172.20.0.1:47470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:58:21,805 - lava_callback - INFO - Received callback for node 69f93ca6d09fe59064f055fa INFO: 172.20.0.1:37120 - "POST /node/69f93ca6d09fe59064f055fa HTTP/1.0" 202 Accepted 2026-05-05 04:58:21,811 - lava_callback - INFO - Starting async job submit for node 69f93ca6d09fe59064f055fa 2026-05-05 04:58:21,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:21,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca6d09fe59064f055fa HTTP/1.1" 200 None 2026-05-05 04:58:21,859 - lava_callback - DEBUG - Processing node 69f93ca6d09fe59064f055fa: ltp-input 2026-05-05 04:58:22,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:58:22,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:58:22,274 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69f93ca6d09fe59064f055fa/log.txt.gz 2026-05-05 04:58:22,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:58:22,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:58:22,624 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69f93ca6d09fe59064f055fa/lava-logs.yaml 2026-05-05 04:58:22,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 04:58:22,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 04:58:22,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69f93ca6d09fe59064f055fa/lava_callback.json.gz 2026-05-05 04:58:22,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:22,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca6d09fe59064f055fa HTTP/1.1" 200 None 2026-05-05 04:58:22,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:22,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 04:58:22,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:22,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ca6d09fe59064f055fa HTTP/1.1" 200 None 2026-05-05 04:58:22,844 - lava_callback - INFO - Completed processing callback for node 69f93ca6d09fe59064f055fa INFO: 172.20.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36850 - "GET / HTTP/1.0" 200 OK 2026-05-05 04:58:44,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 04:58:44,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:00:22,191 - lava_callback - INFO - Received callback for node 69f93ca8d09fe59064f05605 INFO: 172.20.0.1:60780 - "POST /node/69f93ca8d09fe59064f05605 HTTP/1.0" 202 Accepted 2026-05-05 05:00:22,199 - lava_callback - INFO - Starting async job submit for node 69f93ca8d09fe59064f05605 2026-05-05 05:00:22,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:00:22,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca8d09fe59064f05605 HTTP/1.1" 200 None 2026-05-05 05:00:22,242 - lava_callback - DEBUG - Processing node 69f93ca8d09fe59064f05605: ltp-ipc 2026-05-05 05:00:22,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:00:22,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:00:22,735 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69f93ca8d09fe59064f05605/log.txt.gz 2026-05-05 05:00:22,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:00:23,134 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:00:23,136 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69f93ca8d09fe59064f05605/lava-logs.yaml 2026-05-05 05:00:23,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:00:23,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:00:23,306 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69f93ca8d09fe59064f05605/lava_callback.json.gz 2026-05-05 05:00:23,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:00:23,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca8d09fe59064f05605 HTTP/1.1" 200 None 2026-05-05 05:00:23,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:00:23,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 05:00:23,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:00:23,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ca8d09fe59064f05605 HTTP/1.1" 200 None 2026-05-05 05:00:23,386 - lava_callback - INFO - Completed processing callback for node 69f93ca8d09fe59064f05605 INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44242 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:00:44,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:00:44,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53914 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:02:21,873 - lava_callback - INFO - Received callback for node 69f93ca9d09fe59064f05614 INFO: 172.20.0.1:45648 - "POST /node/69f93ca9d09fe59064f05614 HTTP/1.0" 202 Accepted 2026-05-05 05:02:21,879 - lava_callback - INFO - Starting async job submit for node 69f93ca9d09fe59064f05614 2026-05-05 05:02:21,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:02:21,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca9d09fe59064f05614 HTTP/1.1" 200 None 2026-05-05 05:02:21,939 - lava_callback - DEBUG - Processing node 69f93ca9d09fe59064f05614: ltp-pty 2026-05-05 05:02:22,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:02:22,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:02:22,492 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69f93ca9d09fe59064f05614/log.txt.gz 2026-05-05 05:02:22,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:02:22,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:02:22,913 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69f93ca9d09fe59064f05614/lava-logs.yaml 2026-05-05 05:02:22,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:02:23,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:02:23,081 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69f93ca9d09fe59064f05614/lava_callback.json.gz 2026-05-05 05:02:23,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:02:23,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93ca9d09fe59064f05614 HTTP/1.1" 200 None 2026-05-05 05:02:23,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:02:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 05:02:23,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:02:23,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93ca9d09fe59064f05614 HTTP/1.1" 200 None 2026-05-05 05:02:23,178 - lava_callback - INFO - Completed processing callback for node 69f93ca9d09fe59064f05614 INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32932 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:02:44,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:02:44,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57222 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:04:21,950 - lava_callback - INFO - Received callback for node 69f93cabd09fe59064f05617 INFO: 172.20.0.1:34250 - "POST /node/69f93cabd09fe59064f05617 HTTP/1.0" 202 Accepted 2026-05-05 05:04:21,956 - lava_callback - INFO - Starting async job submit for node 69f93cabd09fe59064f05617 2026-05-05 05:04:21,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:04:22,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cabd09fe59064f05617 HTTP/1.1" 200 None 2026-05-05 05:04:22,002 - lava_callback - DEBUG - Processing node 69f93cabd09fe59064f05617: ltp-sched 2026-05-05 05:04:22,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:04:22,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:04:22,487 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69f93cabd09fe59064f05617/log.txt.gz 2026-05-05 05:04:22,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:04:22,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:04:22,889 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69f93cabd09fe59064f05617/lava-logs.yaml 2026-05-05 05:04:22,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:04:23,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:04:23,040 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69f93cabd09fe59064f05617/lava_callback.json.gz 2026-05-05 05:04:23,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:04:23,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cabd09fe59064f05617 HTTP/1.1" 200 None 2026-05-05 05:04:23,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:04:23,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 05:04:23,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:04:23,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93cabd09fe59064f05617 HTTP/1.1" 200 None 2026-05-05 05:04:23,127 - lava_callback - INFO - Completed processing callback for node 69f93cabd09fe59064f05617 INFO: 172.20.0.1:34266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32828 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:04:44,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:04:44,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40072 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:08:19,888 - lava_callback - INFO - Received callback for node 69f6d83b1f903fec0b0e0133 INFO: 172.20.0.1:48006 - "POST /node/69f6d83b1f903fec0b0e0133 HTTP/1.0" 202 Accepted 2026-05-05 05:08:19,894 - lava_callback - INFO - Starting async job submit for node 69f6d83b1f903fec0b0e0133 2026-05-05 05:08:19,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:08:19,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d83b1f903fec0b0e0133 HTTP/1.1" 200 None 2026-05-05 05:08:19,920 - lava_callback - DEBUG - Processing node 69f6d83b1f903fec0b0e0133: baseline-arm64-baylibre 2026-05-05 05:08:19,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d83b1f903fec0b0e0133 2026-05-05 05:08:19,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:08:19,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d83b1f903fec0b0e0133 HTTP/1.1" 200 None 2026-05-05 05:08:19,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:08:19,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623ae8234569b79a1c059 HTTP/1.1" 200 None 2026-05-05 05:08:19,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:08:19,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d83b1f903fec0b0e0133 HTTP/1.1" 200 None 2026-05-05 05:08:19,970 - lava_callback - ERROR - Error processing callback for node 69f6d83b1f903fec0b0e0133: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60540 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:09:14,284 - lava_callback - INFO - Received callback for node 69f93cb0d09fe59064f05636 INFO: 172.20.0.1:32972 - "POST /node/69f93cb0d09fe59064f05636 HTTP/1.0" 202 Accepted 2026-05-05 05:09:14,295 - lava_callback - INFO - Starting async job submit for node 69f93cb0d09fe59064f05636 2026-05-05 05:09:14,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:14,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cb0d09fe59064f05636 HTTP/1.1" 200 None 2026-05-05 05:09:14,336 - lava_callback - DEBUG - Processing node 69f93cb0d09fe59064f05636: ltp-syscalls-ipc 2026-05-05 05:09:14,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:09:15,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:09:15,008 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69f93cb0d09fe59064f05636/log.txt.gz INFO: 172.20.0.1:32974 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 05:09:15,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:09:15,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:09:15,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69f93cb0d09fe59064f05636/lava-logs.yaml 2026-05-05 05:09:15,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:09:15,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:09:15,565 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69f93cb0d09fe59064f05636/lava_callback.json.gz 2026-05-05 05:09:15,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:15,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cb0d09fe59064f05636 HTTP/1.1" 200 None 2026-05-05 05:09:15,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:15,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 05:09:15,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:15,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93cb0d09fe59064f05636 HTTP/1.1" 200 None 2026-05-05 05:09:15,639 - lava_callback - INFO - Completed processing callback for node 69f93cb0d09fe59064f05636 INFO: 172.20.0.1:32978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55174 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:09:41,953 - lava_callback - INFO - Received callback for node 69f93cb2d09fe59064f05639 INFO: 172.20.0.1:35398 - "POST /node/69f93cb2d09fe59064f05639 HTTP/1.0" 202 Accepted 2026-05-05 05:09:41,960 - lava_callback - INFO - Starting async job submit for node 69f93cb2d09fe59064f05639 2026-05-05 05:09:41,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:41,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cb2d09fe59064f05639 HTTP/1.1" 200 None 2026-05-05 05:09:42,000 - lava_callback - DEBUG - Processing node 69f93cb2d09fe59064f05639: ltp-timers 2026-05-05 05:09:42,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:09:42,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:09:42,467 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69f93cb2d09fe59064f05639/log.txt.gz 2026-05-05 05:09:42,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:09:42,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:09:42,836 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69f93cb2d09fe59064f05639/lava-logs.yaml 2026-05-05 05:09:42,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:09:42,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:09:42,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69f93cb2d09fe59064f05639/lava_callback.json.gz 2026-05-05 05:09:42,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:42,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cb2d09fe59064f05639 HTTP/1.1" 200 None 2026-05-05 05:09:42,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:43,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 05:09:43,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:43,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93cb2d09fe59064f05639 HTTP/1.1" 200 None 2026-05-05 05:09:43,040 - lava_callback - INFO - Completed processing callback for node 69f93cb2d09fe59064f05639 2026-05-05 05:09:45,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:09:45,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35266 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:10:34,356 - lava_callback - INFO - Received callback for node 69f6d8ba1f903fec0b0e0164 INFO: 172.20.0.1:35280 - "POST /node/69f6d8ba1f903fec0b0e0164 HTTP/1.0" 202 Accepted 2026-05-05 05:10:34,367 - lava_callback - INFO - Starting async job submit for node 69f6d8ba1f903fec0b0e0164 2026-05-05 05:10:34,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:10:34,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d8ba1f903fec0b0e0164 HTTP/1.1" 200 None 2026-05-05 05:10:34,406 - lava_callback - DEBUG - Processing node 69f6d8ba1f903fec0b0e0164: baseline-arm64-clabbe 2026-05-05 05:10:34,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d8ba1f903fec0b0e0164 2026-05-05 05:10:34,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:10:34,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d8ba1f903fec0b0e0164 HTTP/1.1" 200 None 2026-05-05 05:10:34,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:10:34,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623ae8234569b79a1c059 HTTP/1.1" 200 None 2026-05-05 05:10:34,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:10:34,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d8ba1f903fec0b0e0164 HTTP/1.1" 200 None 2026-05-05 05:10:34,457 - lava_callback - ERROR - Error processing callback for node 69f6d8ba1f903fec0b0e0164: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33764 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:11:42,065 - lava_callback - INFO - Received callback for node 69f93cacd09fe59064f05630 INFO: 172.20.0.1:51096 - "POST /node/69f93cacd09fe59064f05630 HTTP/1.0" 202 Accepted 2026-05-05 05:11:42,071 - lava_callback - INFO - Starting async job submit for node 69f93cacd09fe59064f05630 2026-05-05 05:11:42,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:42,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cacd09fe59064f05630 HTTP/1.1" 200 None 2026-05-05 05:11:42,120 - lava_callback - DEBUG - Processing node 69f93cacd09fe59064f05630: ltp-smoke 2026-05-05 05:11:42,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:11:42,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:11:42,788 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69f93cacd09fe59064f05630/log.txt.gz 2026-05-05 05:11:43,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:11:43,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:11:43,206 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69f93cacd09fe59064f05630/lava-logs.yaml 2026-05-05 05:11:43,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:11:43,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:11:43,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69f93cacd09fe59064f05630/lava_callback.json.gz 2026-05-05 05:11:43,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:43,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cacd09fe59064f05630 HTTP/1.1" 200 None 2026-05-05 05:11:43,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:43,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 05:11:43,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:43,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93cacd09fe59064f05630 HTTP/1.1" 200 None 2026-05-05 05:11:43,608 - lava_callback - INFO - Completed processing callback for node 69f93cacd09fe59064f05630 2026-05-05 05:11:45,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:45,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51110 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:11:54,640 - lava_callback - INFO - Received callback for node 69f6d9081f903fec0b0e019d INFO: 172.20.0.1:37546 - "POST /node/69f6d9081f903fec0b0e019d HTTP/1.0" 202 Accepted 2026-05-05 05:11:54,646 - lava_callback - INFO - Starting async job submit for node 69f6d9081f903fec0b0e019d 2026-05-05 05:11:54,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:54,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d9081f903fec0b0e019d HTTP/1.1" 200 None 2026-05-05 05:11:54,661 - lava_callback - DEBUG - Processing node 69f6d9081f903fec0b0e019d: baseline-arm64-clabbe 2026-05-05 05:11:54,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d9081f903fec0b0e019d 2026-05-05 05:11:54,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:54,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d9081f903fec0b0e019d HTTP/1.1" 200 None 2026-05-05 05:11:54,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:54,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623ae8234569b79a1c059 HTTP/1.1" 200 None 2026-05-05 05:11:54,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:11:54,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d9081f903fec0b0e019d HTTP/1.1" 200 None 2026-05-05 05:11:54,706 - lava_callback - ERROR - Error processing callback for node 69f6d9081f903fec0b0e019d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49086 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:12:41,907 - lava_callback - INFO - Received callback for node 69f94292d09fe59064f05a60 INFO: 172.20.0.1:37984 - "POST /node/69f94292d09fe59064f05a60 HTTP/1.0" 202 Accepted 2026-05-05 05:12:41,913 - lava_callback - INFO - Starting async job submit for node 69f94292d09fe59064f05a60 2026-05-05 05:12:41,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:12:41,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94292d09fe59064f05a60 HTTP/1.1" 200 None 2026-05-05 05:12:41,950 - lava_callback - DEBUG - Processing node 69f94292d09fe59064f05a60: baseline-arm64-broonie 2026-05-05 05:12:42,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:12:42,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:12:42,457 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94292d09fe59064f05a60/log.txt.gz 2026-05-05 05:12:42,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:12:42,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:12:42,766 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94292d09fe59064f05a60/lava-logs.yaml 2026-05-05 05:12:42,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:12:42,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:12:42,878 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f94292d09fe59064f05a60/lava_callback.json.gz 2026-05-05 05:12:42,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:12:42,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f94292d09fe59064f05a60 HTTP/1.1" 200 None 2026-05-05 05:12:42,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:12:42,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 05:12:42,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:12:42,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f94292d09fe59064f05a60 HTTP/1.1" 200 None 2026-05-05 05:12:42,964 - lava_callback - INFO - Completed processing callback for node 69f94292d09fe59064f05a60 2026-05-05 05:12:45,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:12:45,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54438 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:13:14,465 - lava_callback - INFO - Received callback for node 69f6d9541f903fec0b0e01be INFO: 172.20.0.1:43364 - "POST /node/69f6d9541f903fec0b0e01be HTTP/1.0" 202 Accepted 2026-05-05 05:13:14,471 - lava_callback - INFO - Starting async job submit for node 69f6d9541f903fec0b0e01be 2026-05-05 05:13:14,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:14,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d9541f903fec0b0e01be HTTP/1.1" 200 None 2026-05-05 05:13:14,489 - lava_callback - DEBUG - Processing node 69f6d9541f903fec0b0e01be: baseline-arm64-clabbe 2026-05-05 05:13:14,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d9541f903fec0b0e01be 2026-05-05 05:13:14,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:14,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d9541f903fec0b0e01be HTTP/1.1" 200 None 2026-05-05 05:13:14,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:14,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623ae8234569b79a1c059 HTTP/1.1" 200 None 2026-05-05 05:13:14,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:14,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d9541f903fec0b0e01be HTTP/1.1" 200 None 2026-05-05 05:13:14,532 - lava_callback - ERROR - Error processing callback for node 69f6d9541f903fec0b0e01be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 05:13:14,666 - lava_callback - INFO - Received callback for node 69f93cb3d09fe59064f0563c INFO: 172.20.0.1:43374 - "POST /node/69f93cb3d09fe59064f0563c HTTP/1.0" 202 Accepted 2026-05-05 05:13:14,673 - lava_callback - INFO - Starting async job submit for node 69f93cb3d09fe59064f0563c 2026-05-05 05:13:14,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:14,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cb3d09fe59064f0563c HTTP/1.1" 200 None 2026-05-05 05:13:14,723 - lava_callback - DEBUG - Processing node 69f93cb3d09fe59064f0563c: ltp-watchqueue 2026-05-05 05:13:15,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:43378 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 05:13:15,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:13:15,297 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69f93cb3d09fe59064f0563c/log.txt.gz 2026-05-05 05:13:15,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:13:15,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:13:15,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69f93cb3d09fe59064f0563c/lava-logs.yaml 2026-05-05 05:13:15,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 05:13:16,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 05:13:16,014 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69f93cb3d09fe59064f0563c/lava_callback.json.gz 2026-05-05 05:13:16,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:16,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93cb3d09fe59064f0563c HTTP/1.1" 200 None 2026-05-05 05:13:16,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:16,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93702d09fe59064f02ceb HTTP/1.1" 200 None 2026-05-05 05:13:16,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:16,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f93cb3d09fe59064f0563c HTTP/1.1" 200 None 2026-05-05 05:13:16,096 - lava_callback - INFO - Completed processing callback for node 69f93cb3d09fe59064f0563c INFO: 172.20.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41426 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:13:45,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:13:45,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40606 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:14:14,499 - lava_callback - INFO - Received callback for node 69f6d99d1f903fec0b0e01f8 INFO: 172.20.0.1:52386 - "POST /node/69f6d99d1f903fec0b0e01f8 HTTP/1.0" 202 Accepted 2026-05-05 05:14:14,505 - lava_callback - INFO - Starting async job submit for node 69f6d99d1f903fec0b0e01f8 2026-05-05 05:14:14,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:14:14,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d99d1f903fec0b0e01f8 HTTP/1.1" 200 None 2026-05-05 05:14:14,521 - lava_callback - DEBUG - Processing node 69f6d99d1f903fec0b0e01f8: baseline-arm64-clabbe 2026-05-05 05:14:14,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6d99d1f903fec0b0e01f8 2026-05-05 05:14:14,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:14:14,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6d99d1f903fec0b0e01f8 HTTP/1.1" 200 None 2026-05-05 05:14:14,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:14:14,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623ae8234569b79a1c059 HTTP/1.1" 200 None 2026-05-05 05:14:14,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:14:14,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6d99d1f903fec0b0e01f8 HTTP/1.1" 200 None 2026-05-05 05:14:14,573 - lava_callback - ERROR - Error processing callback for node 69f6d99d1f903fec0b0e01f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59212 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:16:54,602 - lava_callback - INFO - Received callback for node 69f6da311f903fec0b0e023d INFO: 172.20.0.1:43394 - "POST /node/69f6da311f903fec0b0e023d HTTP/1.0" 202 Accepted 2026-05-05 05:16:54,607 - lava_callback - INFO - Starting async job submit for node 69f6da311f903fec0b0e023d 2026-05-05 05:16:54,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:16:54,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6da311f903fec0b0e023d HTTP/1.1" 200 None 2026-05-05 05:16:54,622 - lava_callback - DEBUG - Processing node 69f6da311f903fec0b0e023d: baseline-arm64-clabbe 2026-05-05 05:16:54,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6da311f903fec0b0e023d 2026-05-05 05:16:54,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:16:54,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6da311f903fec0b0e023d HTTP/1.1" 200 None 2026-05-05 05:16:54,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:16:54,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623ae8234569b79a1c059 HTTP/1.1" 200 None 2026-05-05 05:16:54,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:16:54,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6da311f903fec0b0e023d HTTP/1.1" 200 None 2026-05-05 05:16:54,675 - lava_callback - ERROR - Error processing callback for node 69f6da311f903fec0b0e023d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55588 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:17:54,615 - lava_callback - INFO - Received callback for node 69f6da7b1f903fec0b0e0270 INFO: 172.20.0.1:40500 - "POST /node/69f6da7b1f903fec0b0e0270 HTTP/1.0" 202 Accepted 2026-05-05 05:17:54,620 - lava_callback - INFO - Starting async job submit for node 69f6da7b1f903fec0b0e0270 2026-05-05 05:17:54,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:17:54,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6da7b1f903fec0b0e0270 HTTP/1.1" 200 None 2026-05-05 05:17:54,636 - lava_callback - DEBUG - Processing node 69f6da7b1f903fec0b0e0270: baseline-arm64-clabbe 2026-05-05 05:17:54,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6da7b1f903fec0b0e0270 2026-05-05 05:17:54,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:17:54,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6da7b1f903fec0b0e0270 HTTP/1.1" 200 None 2026-05-05 05:17:54,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:17:54,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f623ae8234569b79a1c059 HTTP/1.1" 200 None 2026-05-05 05:17:54,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:17:54,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6da7b1f903fec0b0e0270 HTTP/1.1" 200 None 2026-05-05 05:17:54,688 - lava_callback - ERROR - Error processing callback for node 69f6da7b1f903fec0b0e0270: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47004 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:22:40,722 - lava_callback - INFO - Received callback for node 69f6db8e1f903fec0b0e02ab INFO: 172.20.0.1:42244 - "POST /node/69f6db8e1f903fec0b0e02ab HTTP/1.0" 202 Accepted 2026-05-05 05:22:40,727 - lava_callback - INFO - Starting async job submit for node 69f6db8e1f903fec0b0e02ab 2026-05-05 05:22:40,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:22:40,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6db8e1f903fec0b0e02ab HTTP/1.1" 200 None 2026-05-05 05:22:40,744 - lava_callback - DEBUG - Processing node 69f6db8e1f903fec0b0e02ab: baseline-arm64-baylibre 2026-05-05 05:22:40,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6db8e1f903fec0b0e02ab 2026-05-05 05:22:40,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:22:40,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6db8e1f903fec0b0e02ab HTTP/1.1" 200 None 2026-05-05 05:22:40,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:22:40,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:22:40,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:22:40,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6db8e1f903fec0b0e02ab HTTP/1.1" 200 None 2026-05-05 05:22:40,792 - lava_callback - ERROR - Error processing callback for node 69f6db8e1f903fec0b0e02ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:24:14,850 - lava_callback - INFO - Received callback for node 69f6dbe81f903fec0b0e02d9 INFO: 172.20.0.1:52078 - "POST /node/69f6dbe81f903fec0b0e02d9 HTTP/1.0" 202 Accepted 2026-05-05 05:24:14,856 - lava_callback - INFO - Starting async job submit for node 69f6dbe81f903fec0b0e02d9 2026-05-05 05:24:14,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:24:14,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dbe81f903fec0b0e02d9 HTTP/1.1" 200 None 2026-05-05 05:24:14,877 - lava_callback - DEBUG - Processing node 69f6dbe81f903fec0b0e02d9: baseline-arm64-clabbe 2026-05-05 05:24:14,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6dbe81f903fec0b0e02d9 2026-05-05 05:24:14,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:24:14,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dbe81f903fec0b0e02d9 HTTP/1.1" 200 None 2026-05-05 05:24:14,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:24:14,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:24:14,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:24:14,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6dbe81f903fec0b0e02d9 HTTP/1.1" 200 None 2026-05-05 05:24:14,927 - lava_callback - ERROR - Error processing callback for node 69f6dbe81f903fec0b0e02d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36908 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:25:54,963 - lava_callback - INFO - Received callback for node 69f6dc4d1f903fec0b0e0307 INFO: 172.20.0.1:51520 - "POST /node/69f6dc4d1f903fec0b0e0307 HTTP/1.0" 202 Accepted 2026-05-05 05:25:54,968 - lava_callback - INFO - Starting async job submit for node 69f6dc4d1f903fec0b0e0307 2026-05-05 05:25:54,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:25:54,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dc4d1f903fec0b0e0307 HTTP/1.1" 200 None 2026-05-05 05:25:54,984 - lava_callback - DEBUG - Processing node 69f6dc4d1f903fec0b0e0307: baseline-arm64-clabbe 2026-05-05 05:25:54,984 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6dc4d1f903fec0b0e0307 2026-05-05 05:25:54,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:25:54,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dc4d1f903fec0b0e0307 HTTP/1.1" 200 None 2026-05-05 05:25:54,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:25:55,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:25:55,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:25:55,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6dc4d1f903fec0b0e0307 HTTP/1.1" 200 None 2026-05-05 05:25:55,048 - lava_callback - ERROR - Error processing callback for node 69f6dc4d1f903fec0b0e0307: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:27:14,974 - lava_callback - INFO - Received callback for node 69f6dc9d1f903fec0b0e0335 INFO: 172.20.0.1:59728 - "POST /node/69f6dc9d1f903fec0b0e0335 HTTP/1.0" 202 Accepted 2026-05-05 05:27:14,979 - lava_callback - INFO - Starting async job submit for node 69f6dc9d1f903fec0b0e0335 2026-05-05 05:27:14,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:27:14,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dc9d1f903fec0b0e0335 HTTP/1.1" 200 None 2026-05-05 05:27:14,995 - lava_callback - DEBUG - Processing node 69f6dc9d1f903fec0b0e0335: baseline-arm64-clabbe 2026-05-05 05:27:14,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6dc9d1f903fec0b0e0335 2026-05-05 05:27:14,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:27:15,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dc9d1f903fec0b0e0335 HTTP/1.1" 200 None 2026-05-05 05:27:15,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:27:15,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:27:15,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:27:15,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6dc9d1f903fec0b0e0335 HTTP/1.1" 200 None 2026-05-05 05:27:15,046 - lava_callback - ERROR - Error processing callback for node 69f6dc9d1f903fec0b0e0335: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43920 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:28:34,967 - lava_callback - INFO - Received callback for node 69f6dced1f903fec0b0e036b INFO: 172.20.0.1:43934 - "POST /node/69f6dced1f903fec0b0e036b HTTP/1.0" 202 Accepted 2026-05-05 05:28:34,972 - lava_callback - INFO - Starting async job submit for node 69f6dced1f903fec0b0e036b 2026-05-05 05:28:34,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:28:34,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dced1f903fec0b0e036b HTTP/1.1" 200 None 2026-05-05 05:28:34,988 - lava_callback - DEBUG - Processing node 69f6dced1f903fec0b0e036b: baseline-arm64-clabbe 2026-05-05 05:28:34,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6dced1f903fec0b0e036b 2026-05-05 05:28:34,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:28:35,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dced1f903fec0b0e036b HTTP/1.1" 200 None 2026-05-05 05:28:35,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:28:35,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:28:35,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:28:35,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6dced1f903fec0b0e036b HTTP/1.1" 200 None 2026-05-05 05:28:35,039 - lava_callback - ERROR - Error processing callback for node 69f6dced1f903fec0b0e036b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39482 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:30:55,089 - lava_callback - INFO - Received callback for node 69f6dd831f903fec0b0e03c8 INFO: 172.20.0.1:41712 - "POST /node/69f6dd831f903fec0b0e03c8 HTTP/1.0" 202 Accepted 2026-05-05 05:30:55,095 - lava_callback - INFO - Starting async job submit for node 69f6dd831f903fec0b0e03c8 2026-05-05 05:30:55,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:30:55,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dd831f903fec0b0e03c8 HTTP/1.1" 200 None 2026-05-05 05:30:55,110 - lava_callback - DEBUG - Processing node 69f6dd831f903fec0b0e03c8: baseline-arm64-clabbe 2026-05-05 05:30:55,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6dd831f903fec0b0e03c8 2026-05-05 05:30:55,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:30:55,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dd831f903fec0b0e03c8 HTTP/1.1" 200 None 2026-05-05 05:30:55,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:30:55,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:30:55,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:30:55,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6dd831f903fec0b0e03c8 HTTP/1.1" 200 None 2026-05-05 05:30:55,156 - lava_callback - ERROR - Error processing callback for node 69f6dd831f903fec0b0e03c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60530 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:32:15,124 - lava_callback - INFO - Received callback for node 69f6ddcd1f903fec0b0e03ee INFO: 172.20.0.1:48928 - "POST /node/69f6ddcd1f903fec0b0e03ee HTTP/1.0" 202 Accepted 2026-05-05 05:32:15,130 - lava_callback - INFO - Starting async job submit for node 69f6ddcd1f903fec0b0e03ee 2026-05-05 05:32:15,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:32:15,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ddcd1f903fec0b0e03ee HTTP/1.1" 200 None 2026-05-05 05:32:15,145 - lava_callback - DEBUG - Processing node 69f6ddcd1f903fec0b0e03ee: baseline-arm64-clabbe 2026-05-05 05:32:15,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ddcd1f903fec0b0e03ee 2026-05-05 05:32:15,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:32:15,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ddcd1f903fec0b0e03ee HTTP/1.1" 200 None 2026-05-05 05:32:15,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:48930 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 05:32:15,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:32:15,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:32:15,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ddcd1f903fec0b0e03ee HTTP/1.1" 200 None 2026-05-05 05:32:15,201 - lava_callback - ERROR - Error processing callback for node 69f6ddcd1f903fec0b0e03ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50448 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:39:21,621 - lava_callback - INFO - Received callback for node 69f6df831f903fec0b0e0437 INFO: 172.20.0.1:49850 - "POST /node/69f6df831f903fec0b0e0437 HTTP/1.0" 202 Accepted 2026-05-05 05:39:21,627 - lava_callback - INFO - Starting async job submit for node 69f6df831f903fec0b0e0437 2026-05-05 05:39:21,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:39:21,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6df831f903fec0b0e0437 HTTP/1.1" 200 None 2026-05-05 05:39:21,643 - lava_callback - DEBUG - Processing node 69f6df831f903fec0b0e0437: baseline-arm64-baylibre 2026-05-05 05:39:21,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6df831f903fec0b0e0437 2026-05-05 05:39:21,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:39:21,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6df831f903fec0b0e0437 HTTP/1.1" 200 None 2026-05-05 05:39:21,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:39:21,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:39:21,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:39:21,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6df831f903fec0b0e0437 HTTP/1.1" 200 None 2026-05-05 05:39:21,697 - lava_callback - ERROR - Error processing callback for node 69f6df831f903fec0b0e0437: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44780 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:41:35,504 - lava_callback - INFO - Received callback for node 69f6dfff1f903fec0b0e0463 INFO: 172.20.0.1:44782 - "POST /node/69f6dfff1f903fec0b0e0463 HTTP/1.0" 202 Accepted 2026-05-05 05:41:35,510 - lava_callback - INFO - Starting async job submit for node 69f6dfff1f903fec0b0e0463 2026-05-05 05:41:35,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:41:35,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dfff1f903fec0b0e0463 HTTP/1.1" 200 None 2026-05-05 05:41:35,525 - lava_callback - DEBUG - Processing node 69f6dfff1f903fec0b0e0463: baseline-arm64-clabbe 2026-05-05 05:41:35,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6dfff1f903fec0b0e0463 2026-05-05 05:41:35,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:41:35,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6dfff1f903fec0b0e0463 HTTP/1.1" 200 None 2026-05-05 05:41:35,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:41:35,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:41:35,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:41:35,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6dfff1f903fec0b0e0463 HTTP/1.1" 200 None 2026-05-05 05:41:35,573 - lava_callback - ERROR - Error processing callback for node 69f6dfff1f903fec0b0e0463: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52830 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:42:55,561 - lava_callback - INFO - Received callback for node 69f6e04c1f903fec0b0e049d INFO: 172.20.0.1:43048 - "POST /node/69f6e04c1f903fec0b0e049d HTTP/1.0" 202 Accepted 2026-05-05 05:42:55,567 - lava_callback - INFO - Starting async job submit for node 69f6e04c1f903fec0b0e049d 2026-05-05 05:42:55,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:42:55,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e04c1f903fec0b0e049d HTTP/1.1" 200 None 2026-05-05 05:42:55,585 - lava_callback - DEBUG - Processing node 69f6e04c1f903fec0b0e049d: baseline-arm64-clabbe 2026-05-05 05:42:55,586 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e04c1f903fec0b0e049d 2026-05-05 05:42:55,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:42:55,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e04c1f903fec0b0e049d HTTP/1.1" 200 None 2026-05-05 05:42:55,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:42:55,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:42:55,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:42:55,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e04c1f903fec0b0e049d HTTP/1.1" 200 None 2026-05-05 05:42:55,634 - lava_callback - ERROR - Error processing callback for node 69f6e04c1f903fec0b0e049d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36844 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:43:55,641 - lava_callback - INFO - Received callback for node 69f6e0951f903fec0b0e04bb INFO: 172.20.0.1:52096 - "POST /node/69f6e0951f903fec0b0e04bb HTTP/1.0" 202 Accepted 2026-05-05 05:43:55,646 - lava_callback - INFO - Starting async job submit for node 69f6e0951f903fec0b0e04bb 2026-05-05 05:43:55,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:43:55,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e0951f903fec0b0e04bb HTTP/1.1" 200 None 2026-05-05 05:43:55,669 - lava_callback - DEBUG - Processing node 69f6e0951f903fec0b0e04bb: baseline-arm64-clabbe 2026-05-05 05:43:55,669 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e0951f903fec0b0e04bb 2026-05-05 05:43:55,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:43:55,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e0951f903fec0b0e04bb HTTP/1.1" 200 None 2026-05-05 05:43:55,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:43:55,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:43:55,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:43:55,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e0951f903fec0b0e04bb HTTP/1.1" 200 None 2026-05-05 05:43:55,717 - lava_callback - ERROR - Error processing callback for node 69f6e0951f903fec0b0e04bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:45:02,019 - lava_callback - INFO - Received callback for node 69f6e0da1f903fec0b0e04e1 INFO: 172.20.0.1:36804 - "POST /node/69f6e0da1f903fec0b0e04e1 HTTP/1.0" 202 Accepted 2026-05-05 05:45:02,024 - lava_callback - INFO - Starting async job submit for node 69f6e0da1f903fec0b0e04e1 2026-05-05 05:45:02,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:45:02,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e0da1f903fec0b0e04e1 HTTP/1.1" 200 None 2026-05-05 05:45:02,043 - lava_callback - DEBUG - Processing node 69f6e0da1f903fec0b0e04e1: baseline-arm64-baylibre 2026-05-05 05:45:02,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e0da1f903fec0b0e04e1 2026-05-05 05:45:02,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:45:02,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e0da1f903fec0b0e04e1 HTTP/1.1" 200 None 2026-05-05 05:45:02,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:45:02,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:45:02,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:45:02,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e0da1f903fec0b0e04e1 HTTP/1.1" 200 None 2026-05-05 05:45:02,096 - lava_callback - ERROR - Error processing callback for node 69f6e0da1f903fec0b0e04e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38114 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:46:35,683 - lava_callback - INFO - Received callback for node 69f6e1271f903fec0b0e0513 INFO: 172.20.0.1:38128 - "POST /node/69f6e1271f903fec0b0e0513 HTTP/1.0" 202 Accepted 2026-05-05 05:46:35,689 - lava_callback - INFO - Starting async job submit for node 69f6e1271f903fec0b0e0513 2026-05-05 05:46:35,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:46:35,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e1271f903fec0b0e0513 HTTP/1.1" 200 None 2026-05-05 05:46:35,704 - lava_callback - DEBUG - Processing node 69f6e1271f903fec0b0e0513: baseline-arm64-clabbe 2026-05-05 05:46:35,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e1271f903fec0b0e0513 2026-05-05 05:46:35,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:46:35,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e1271f903fec0b0e0513 HTTP/1.1" 200 None 2026-05-05 05:46:35,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:46:35,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:46:35,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:46:35,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e1271f903fec0b0e0513 HTTP/1.1" 200 None 2026-05-05 05:46:35,750 - lava_callback - ERROR - Error processing callback for node 69f6e1271f903fec0b0e0513: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50554 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:47:35,774 - lava_callback - INFO - Received callback for node 69f6e1711f903fec0b0e0537 INFO: 172.20.0.1:50558 - "POST /node/69f6e1711f903fec0b0e0537 HTTP/1.0" 202 Accepted 2026-05-05 05:47:35,780 - lava_callback - INFO - Starting async job submit for node 69f6e1711f903fec0b0e0537 2026-05-05 05:47:35,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:47:35,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e1711f903fec0b0e0537 HTTP/1.1" 200 None 2026-05-05 05:47:35,795 - lava_callback - DEBUG - Processing node 69f6e1711f903fec0b0e0537: baseline-arm64-clabbe 2026-05-05 05:47:35,796 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e1711f903fec0b0e0537 2026-05-05 05:47:35,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:47:35,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e1711f903fec0b0e0537 HTTP/1.1" 200 None 2026-05-05 05:47:35,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:47:35,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:47:35,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:47:35,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e1711f903fec0b0e0537 HTTP/1.1" 200 None 2026-05-05 05:47:35,851 - lava_callback - ERROR - Error processing callback for node 69f6e1711f903fec0b0e0537: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40588 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:48:55,820 - lava_callback - INFO - Received callback for node 69f6e1bb1f903fec0b0e0569 INFO: 172.20.0.1:46198 - "POST /node/69f6e1bb1f903fec0b0e0569 HTTP/1.0" 202 Accepted 2026-05-05 05:48:55,825 - lava_callback - INFO - Starting async job submit for node 69f6e1bb1f903fec0b0e0569 2026-05-05 05:48:55,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:48:55,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e1bb1f903fec0b0e0569 HTTP/1.1" 200 None 2026-05-05 05:48:55,841 - lava_callback - DEBUG - Processing node 69f6e1bb1f903fec0b0e0569: baseline-arm64-clabbe 2026-05-05 05:48:55,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e1bb1f903fec0b0e0569 2026-05-05 05:48:55,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:48:55,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e1bb1f903fec0b0e0569 HTTP/1.1" 200 None 2026-05-05 05:48:55,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:48:55,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:48:55,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:48:55,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e1bb1f903fec0b0e0569 HTTP/1.1" 200 None 2026-05-05 05:48:55,898 - lava_callback - ERROR - Error processing callback for node 69f6e1bb1f903fec0b0e0569: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34686 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 05:51:15,915 - lava_callback - INFO - Received callback for node 69f6e24f1f903fec0b0e05bf INFO: 172.20.0.1:34690 - "POST /node/69f6e24f1f903fec0b0e05bf HTTP/1.0" 202 Accepted 2026-05-05 05:51:15,921 - lava_callback - INFO - Starting async job submit for node 69f6e24f1f903fec0b0e05bf 2026-05-05 05:51:15,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:51:15,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e24f1f903fec0b0e05bf HTTP/1.1" 200 None 2026-05-05 05:51:15,946 - lava_callback - DEBUG - Processing node 69f6e24f1f903fec0b0e05bf: baseline-arm64-clabbe 2026-05-05 05:51:15,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e24f1f903fec0b0e05bf 2026-05-05 05:51:15,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:51:15,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e24f1f903fec0b0e05bf HTTP/1.1" 200 None 2026-05-05 05:51:15,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:51:15,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:51:15,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:51:15,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e24f1f903fec0b0e05bf HTTP/1.1" 200 None 2026-05-05 05:51:16,000 - lava_callback - ERROR - Error processing callback for node 69f6e24f1f903fec0b0e05bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56584 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:52:36,060 - lava_callback - INFO - Received callback for node 69f6e2991f903fec0b0e05dd INFO: 172.20.0.1:56592 - "POST /node/69f6e2991f903fec0b0e05dd HTTP/1.0" 202 Accepted 2026-05-05 05:52:36,065 - lava_callback - INFO - Starting async job submit for node 69f6e2991f903fec0b0e05dd 2026-05-05 05:52:36,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:52:36,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e2991f903fec0b0e05dd HTTP/1.1" 200 None 2026-05-05 05:52:36,085 - lava_callback - DEBUG - Processing node 69f6e2991f903fec0b0e05dd: baseline-arm64-clabbe 2026-05-05 05:52:36,085 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e2991f903fec0b0e05dd 2026-05-05 05:52:36,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:52:36,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e2991f903fec0b0e05dd HTTP/1.1" 200 None 2026-05-05 05:52:36,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:52:36,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:52:36,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:52:36,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e2991f903fec0b0e05dd HTTP/1.1" 200 None 2026-05-05 05:52:36,131 - lava_callback - ERROR - Error processing callback for node 69f6e2991f903fec0b0e05dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:53:56,034 - lava_callback - INFO - Received callback for node 69f6e2e31f903fec0b0e060d INFO: 172.20.0.1:49092 - "POST /node/69f6e2e31f903fec0b0e060d HTTP/1.0" 202 Accepted 2026-05-05 05:53:56,040 - lava_callback - INFO - Starting async job submit for node 69f6e2e31f903fec0b0e060d 2026-05-05 05:53:56,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:53:56,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e2e31f903fec0b0e060d HTTP/1.1" 200 None 2026-05-05 05:53:56,055 - lava_callback - DEBUG - Processing node 69f6e2e31f903fec0b0e060d: baseline-arm64-clabbe 2026-05-05 05:53:56,056 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e2e31f903fec0b0e060d 2026-05-05 05:53:56,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:53:56,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e2e31f903fec0b0e060d HTTP/1.1" 200 None 2026-05-05 05:53:56,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:53:56,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:53:56,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:53:56,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e2e31f903fec0b0e060d HTTP/1.1" 200 None 2026-05-05 05:53:56,154 - lava_callback - ERROR - Error processing callback for node 69f6e2e31f903fec0b0e060d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38518 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:54:56,145 - lava_callback - INFO - Received callback for node 69f6e32d1f903fec0b0e062d INFO: 172.20.0.1:44036 - "POST /node/69f6e32d1f903fec0b0e062d HTTP/1.0" 202 Accepted 2026-05-05 05:54:56,151 - lava_callback - INFO - Starting async job submit for node 69f6e32d1f903fec0b0e062d 2026-05-05 05:54:56,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:54:56,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e32d1f903fec0b0e062d HTTP/1.1" 200 None 2026-05-05 05:54:56,173 - lava_callback - DEBUG - Processing node 69f6e32d1f903fec0b0e062d: baseline-arm64-clabbe 2026-05-05 05:54:56,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e32d1f903fec0b0e062d 2026-05-05 05:54:56,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:54:56,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e32d1f903fec0b0e062d HTTP/1.1" 200 None 2026-05-05 05:54:56,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:54:56,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:54:56,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:54:56,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e32d1f903fec0b0e062d HTTP/1.1" 200 None 2026-05-05 05:54:56,223 - lava_callback - ERROR - Error processing callback for node 69f6e32d1f903fec0b0e062d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32944 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:57:36,158 - lava_callback - INFO - Received callback for node 69f6e3c21f903fec0b0e067d INFO: 172.20.0.1:32958 - "POST /node/69f6e3c21f903fec0b0e067d HTTP/1.0" 202 Accepted 2026-05-05 05:57:36,164 - lava_callback - INFO - Starting async job submit for node 69f6e3c21f903fec0b0e067d 2026-05-05 05:57:36,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:57:36,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e3c21f903fec0b0e067d HTTP/1.1" 200 None 2026-05-05 05:57:36,190 - lava_callback - DEBUG - Processing node 69f6e3c21f903fec0b0e067d: baseline-arm64-clabbe 2026-05-05 05:57:36,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e3c21f903fec0b0e067d 2026-05-05 05:57:36,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:57:36,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e3c21f903fec0b0e067d HTTP/1.1" 200 None 2026-05-05 05:57:36,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:57:36,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:57:36,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:57:36,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e3c21f903fec0b0e067d HTTP/1.1" 200 None 2026-05-05 05:57:36,247 - lava_callback - ERROR - Error processing callback for node 69f6e3c21f903fec0b0e067d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51128 - "GET / HTTP/1.0" 200 OK 2026-05-05 05:58:56,218 - lava_callback - INFO - Received callback for node 69f6e40d1f903fec0b0e06af INFO: 172.20.0.1:53120 - "POST /node/69f6e40d1f903fec0b0e06af HTTP/1.0" 202 Accepted 2026-05-05 05:58:56,224 - lava_callback - INFO - Starting async job submit for node 69f6e40d1f903fec0b0e06af 2026-05-05 05:58:56,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:58:56,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e40d1f903fec0b0e06af HTTP/1.1" 200 None 2026-05-05 05:58:56,241 - lava_callback - DEBUG - Processing node 69f6e40d1f903fec0b0e06af: baseline-arm64-clabbe 2026-05-05 05:58:56,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6e40d1f903fec0b0e06af 2026-05-05 05:58:56,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:58:56,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6e40d1f903fec0b0e06af HTTP/1.1" 200 None 2026-05-05 05:58:56,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:58:56,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380b82fa6ff8e8ca8e0a6 HTTP/1.1" 200 None 2026-05-05 05:58:56,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 05:58:56,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6e40d1f903fec0b0e06af HTTP/1.1" 200 None 2026-05-05 05:58:56,297 - lava_callback - ERROR - Error processing callback for node 69f6e40d1f903fec0b0e06af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35452 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:27:17,359 - lava_callback - INFO - Received callback for node 69f6eabf1f903fec0b0e0701 INFO: 172.20.0.1:35456 - "POST /node/69f6eabf1f903fec0b0e0701 HTTP/1.0" 202 Accepted 2026-05-05 06:27:17,364 - lava_callback - INFO - Starting async job submit for node 69f6eabf1f903fec0b0e0701 2026-05-05 06:27:17,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:27:17,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eabf1f903fec0b0e0701 HTTP/1.1" 200 None 2026-05-05 06:27:17,381 - lava_callback - DEBUG - Processing node 69f6eabf1f903fec0b0e0701: baseline-arm-clabbe 2026-05-05 06:27:17,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6eabf1f903fec0b0e0701 2026-05-05 06:27:17,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:27:17,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eabf1f903fec0b0e0701 HTTP/1.1" 200 None 2026-05-05 06:27:17,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:27:17,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:27:17,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:27:17,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6eabf1f903fec0b0e0701 HTTP/1.1" 200 None 2026-05-05 06:27:17,437 - lava_callback - ERROR - Error processing callback for node 69f6eabf1f903fec0b0e0701: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55400 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:29:37,407 - lava_callback - INFO - Received callback for node 69f6eb441f903fec0b0e072b INFO: 172.20.0.1:55414 - "POST /node/69f6eb441f903fec0b0e072b HTTP/1.0" 202 Accepted 2026-05-05 06:29:37,413 - lava_callback - INFO - Starting async job submit for node 69f6eb441f903fec0b0e072b 2026-05-05 06:29:37,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:29:37,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eb441f903fec0b0e072b HTTP/1.1" 200 None 2026-05-05 06:29:37,431 - lava_callback - DEBUG - Processing node 69f6eb441f903fec0b0e072b: baseline-arm-clabbe 2026-05-05 06:29:37,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6eb441f903fec0b0e072b 2026-05-05 06:29:37,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:29:37,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eb441f903fec0b0e072b HTTP/1.1" 200 None 2026-05-05 06:29:37,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:29:37,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:29:37,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:29:37,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6eb441f903fec0b0e072b HTTP/1.1" 200 None 2026-05-05 06:29:37,477 - lava_callback - ERROR - Error processing callback for node 69f6eb441f903fec0b0e072b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36282 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:30:17,407 - lava_callback - INFO - Received callback for node 69f6eb691f903fec0b0e0738 INFO: 172.20.0.1:36298 - "POST /node/69f6eb691f903fec0b0e0738 HTTP/1.0" 202 Accepted 2026-05-05 06:30:17,413 - lava_callback - INFO - Starting async job submit for node 69f6eb691f903fec0b0e0738 2026-05-05 06:30:17,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:30:17,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eb691f903fec0b0e0738 HTTP/1.1" 200 None 2026-05-05 06:30:17,427 - lava_callback - DEBUG - Processing node 69f6eb691f903fec0b0e0738: baseline-arm-clabbe 2026-05-05 06:30:17,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6eb691f903fec0b0e0738 2026-05-05 06:30:17,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:30:17,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eb691f903fec0b0e0738 HTTP/1.1" 200 None 2026-05-05 06:30:17,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:30:17,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:30:17,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:30:17,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6eb691f903fec0b0e0738 HTTP/1.1" 200 None 2026-05-05 06:30:17,489 - lava_callback - ERROR - Error processing callback for node 69f6eb691f903fec0b0e0738: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49762 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:31:37,507 - lava_callback - INFO - Received callback for node 69f6ebb41f903fec0b0e075e INFO: 172.20.0.1:49768 - "POST /node/69f6ebb41f903fec0b0e075e HTTP/1.0" 202 Accepted 2026-05-05 06:31:37,513 - lava_callback - INFO - Starting async job submit for node 69f6ebb41f903fec0b0e075e 2026-05-05 06:31:37,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:37,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ebb41f903fec0b0e075e HTTP/1.1" 200 None 2026-05-05 06:31:37,528 - lava_callback - DEBUG - Processing node 69f6ebb41f903fec0b0e075e: baseline-arm-clabbe 2026-05-05 06:31:37,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ebb41f903fec0b0e075e 2026-05-05 06:31:37,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:37,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ebb41f903fec0b0e075e HTTP/1.1" 200 None 2026-05-05 06:31:37,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:37,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:31:37,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:37,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ebb41f903fec0b0e075e HTTP/1.1" 200 None 2026-05-05 06:31:37,576 - lava_callback - ERROR - Error processing callback for node 69f6ebb41f903fec0b0e075e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59780 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:31:57,455 - lava_callback - INFO - Received callback for node 69f6ebd81f903fec0b0e0769 INFO: 172.20.0.1:59782 - "POST /node/69f6ebd81f903fec0b0e0769 HTTP/1.0" 202 Accepted 2026-05-05 06:31:57,460 - lava_callback - INFO - Starting async job submit for node 69f6ebd81f903fec0b0e0769 2026-05-05 06:31:57,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:57,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ebd81f903fec0b0e0769 HTTP/1.1" 200 None 2026-05-05 06:31:57,489 - lava_callback - DEBUG - Processing node 69f6ebd81f903fec0b0e0769: baseline-arm-clabbe 2026-05-05 06:31:57,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ebd81f903fec0b0e0769 2026-05-05 06:31:57,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:57,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ebd81f903fec0b0e0769 HTTP/1.1" 200 None 2026-05-05 06:31:57,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:57,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:31:57,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:31:57,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ebd81f903fec0b0e0769 HTTP/1.1" 200 None 2026-05-05 06:31:57,543 - lava_callback - ERROR - Error processing callback for node 69f6ebd81f903fec0b0e0769: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56318 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:33:57,664 - lava_callback - INFO - Received callback for node 69f6ec461f903fec0b0e079a INFO: 172.20.0.1:56330 - "POST /node/69f6ec461f903fec0b0e079a HTTP/1.0" 202 Accepted 2026-05-05 06:33:57,670 - lava_callback - INFO - Starting async job submit for node 69f6ec461f903fec0b0e079a 2026-05-05 06:33:57,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:33:57,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ec461f903fec0b0e079a HTTP/1.1" 200 None 2026-05-05 06:33:57,685 - lava_callback - DEBUG - Processing node 69f6ec461f903fec0b0e079a: baseline-arm-clabbe 2026-05-05 06:33:57,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ec461f903fec0b0e079a 2026-05-05 06:33:57,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:33:57,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ec461f903fec0b0e079a HTTP/1.1" 200 None 2026-05-05 06:33:57,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:33:57,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:33:57,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:33:57,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ec461f903fec0b0e079a HTTP/1.1" 200 None 2026-05-05 06:33:57,729 - lava_callback - ERROR - Error processing callback for node 69f6ec461f903fec0b0e079a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:34:37,582 - lava_callback - INFO - Received callback for node 69f6ec6d1f903fec0b0e07a8 INFO: 172.20.0.1:48102 - "POST /node/69f6ec6d1f903fec0b0e07a8 HTTP/1.0" 202 Accepted 2026-05-05 06:34:37,588 - lava_callback - INFO - Starting async job submit for node 69f6ec6d1f903fec0b0e07a8 2026-05-05 06:34:37,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:34:37,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ec6d1f903fec0b0e07a8 HTTP/1.1" 200 None 2026-05-05 06:34:37,604 - lava_callback - DEBUG - Processing node 69f6ec6d1f903fec0b0e07a8: baseline-arm-clabbe 2026-05-05 06:34:37,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ec6d1f903fec0b0e07a8 2026-05-05 06:34:37,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:34:37,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ec6d1f903fec0b0e07a8 HTTP/1.1" 200 None 2026-05-05 06:34:37,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:34:37,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:34:37,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:34:37,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ec6d1f903fec0b0e07a8 HTTP/1.1" 200 None 2026-05-05 06:34:37,656 - lava_callback - ERROR - Error processing callback for node 69f6ec6d1f903fec0b0e07a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48712 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:35:17,664 - lava_callback - INFO - Received callback for node 69f6ec921f903fec0b0e07b6 INFO: 172.20.0.1:48718 - "POST /node/69f6ec921f903fec0b0e07b6 HTTP/1.0" 202 Accepted 2026-05-05 06:35:17,670 - lava_callback - INFO - Starting async job submit for node 69f6ec921f903fec0b0e07b6 2026-05-05 06:35:17,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:35:17,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ec921f903fec0b0e07b6 HTTP/1.1" 200 None 2026-05-05 06:35:17,685 - lava_callback - DEBUG - Processing node 69f6ec921f903fec0b0e07b6: baseline-arm-clabbe 2026-05-05 06:35:17,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ec921f903fec0b0e07b6 2026-05-05 06:35:17,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:35:17,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ec921f903fec0b0e07b6 HTTP/1.1" 200 None 2026-05-05 06:35:17,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:35:17,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:35:17,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:35:17,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ec921f903fec0b0e07b6 HTTP/1.1" 200 None 2026-05-05 06:35:17,754 - lava_callback - ERROR - Error processing callback for node 69f6ec921f903fec0b0e07b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34568 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:36:17,767 - lava_callback - INFO - Received callback for node 69f6ecdb1f903fec0b0e07d6 INFO: 172.20.0.1:34580 - "POST /node/69f6ecdb1f903fec0b0e07d6 HTTP/1.0" 202 Accepted 2026-05-05 06:36:17,772 - lava_callback - INFO - Starting async job submit for node 69f6ecdb1f903fec0b0e07d6 2026-05-05 06:36:17,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:36:17,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ecdb1f903fec0b0e07d6 HTTP/1.1" 200 None 2026-05-05 06:36:17,788 - lava_callback - DEBUG - Processing node 69f6ecdb1f903fec0b0e07d6: baseline-arm-clabbe 2026-05-05 06:36:17,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ecdb1f903fec0b0e07d6 2026-05-05 06:36:17,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:36:17,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ecdb1f903fec0b0e07d6 HTTP/1.1" 200 None 2026-05-05 06:36:17,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:36:17,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:36:17,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:36:17,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ecdb1f903fec0b0e07d6 HTTP/1.1" 200 None 2026-05-05 06:36:17,882 - lava_callback - ERROR - Error processing callback for node 69f6ecdb1f903fec0b0e07d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:37:37,751 - lava_callback - INFO - Received callback for node 69f6ed251f903fec0b0e07eb INFO: 172.20.0.1:50632 - "POST /node/69f6ed251f903fec0b0e07eb HTTP/1.0" 202 Accepted 2026-05-05 06:37:37,759 - lava_callback - INFO - Starting async job submit for node 69f6ed251f903fec0b0e07eb 2026-05-05 06:37:37,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:37:37,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ed251f903fec0b0e07eb HTTP/1.1" 200 None 2026-05-05 06:37:37,779 - lava_callback - DEBUG - Processing node 69f6ed251f903fec0b0e07eb: baseline-arm-clabbe 2026-05-05 06:37:37,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ed251f903fec0b0e07eb 2026-05-05 06:37:37,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:37:37,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ed251f903fec0b0e07eb HTTP/1.1" 200 None 2026-05-05 06:37:37,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:37:37,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:37:37,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:37:37,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ed251f903fec0b0e07eb HTTP/1.1" 200 None 2026-05-05 06:37:37,838 - lava_callback - ERROR - Error processing callback for node 69f6ed251f903fec0b0e07eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56574 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:39:57,796 - lava_callback - INFO - Received callback for node 69f6edb91f903fec0b0e081d INFO: 172.20.0.1:56578 - "POST /node/69f6edb91f903fec0b0e081d HTTP/1.0" 202 Accepted 2026-05-05 06:39:57,801 - lava_callback - INFO - Starting async job submit for node 69f6edb91f903fec0b0e081d 2026-05-05 06:39:57,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:39:57,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6edb91f903fec0b0e081d HTTP/1.1" 200 None 2026-05-05 06:39:57,817 - lava_callback - DEBUG - Processing node 69f6edb91f903fec0b0e081d: baseline-arm-clabbe 2026-05-05 06:39:57,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6edb91f903fec0b0e081d 2026-05-05 06:39:57,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:39:57,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6edb91f903fec0b0e081d HTTP/1.1" 200 None 2026-05-05 06:39:57,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:39:57,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:39:57,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:39:57,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6edb91f903fec0b0e081d HTTP/1.1" 200 None 2026-05-05 06:39:57,909 - lava_callback - ERROR - Error processing callback for node 69f6edb91f903fec0b0e081d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47124 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:40:37,831 - lava_callback - INFO - Received callback for node 69f6edde1f903fec0b0e082a INFO: 172.20.0.1:47132 - "POST /node/69f6edde1f903fec0b0e082a HTTP/1.0" 202 Accepted 2026-05-05 06:40:37,838 - lava_callback - INFO - Starting async job submit for node 69f6edde1f903fec0b0e082a 2026-05-05 06:40:37,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:40:37,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6edde1f903fec0b0e082a HTTP/1.1" 200 None 2026-05-05 06:40:37,853 - lava_callback - DEBUG - Processing node 69f6edde1f903fec0b0e082a: baseline-arm-clabbe 2026-05-05 06:40:37,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6edde1f903fec0b0e082a 2026-05-05 06:40:37,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:40:37,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6edde1f903fec0b0e082a HTTP/1.1" 200 None 2026-05-05 06:40:37,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:40:37,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:40:37,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:40:37,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6edde1f903fec0b0e082a HTTP/1.1" 200 None 2026-05-05 06:40:37,904 - lava_callback - ERROR - Error processing callback for node 69f6edde1f903fec0b0e082a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33452 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:41:17,910 - lava_callback - INFO - Received callback for node 69f6ee031f903fec0b0e0854 INFO: 172.20.0.1:33464 - "POST /node/69f6ee031f903fec0b0e0854 HTTP/1.0" 202 Accepted 2026-05-05 06:41:17,916 - lava_callback - INFO - Starting async job submit for node 69f6ee031f903fec0b0e0854 2026-05-05 06:41:17,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:17,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ee031f903fec0b0e0854 HTTP/1.1" 200 None 2026-05-05 06:41:17,950 - lava_callback - DEBUG - Processing node 69f6ee031f903fec0b0e0854: baseline-arm-clabbe 2026-05-05 06:41:17,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ee031f903fec0b0e0854 2026-05-05 06:41:17,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:17,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ee031f903fec0b0e0854 HTTP/1.1" 200 None 2026-05-05 06:41:17,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:18,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:41:18,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:18,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ee031f903fec0b0e0854 HTTP/1.1" 200 None 2026-05-05 06:41:18,033 - lava_callback - ERROR - Error processing callback for node 69f6ee031f903fec0b0e0854: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51564 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:41:57,856 - lava_callback - INFO - Received callback for node 69f6ee291f903fec0b0e086b INFO: 172.20.0.1:51572 - "POST /node/69f6ee291f903fec0b0e086b HTTP/1.0" 202 Accepted 2026-05-05 06:41:57,861 - lava_callback - INFO - Starting async job submit for node 69f6ee291f903fec0b0e086b 2026-05-05 06:41:57,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:57,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ee291f903fec0b0e086b HTTP/1.1" 200 None 2026-05-05 06:41:57,877 - lava_callback - DEBUG - Processing node 69f6ee291f903fec0b0e086b: baseline-arm-clabbe 2026-05-05 06:41:57,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ee291f903fec0b0e086b 2026-05-05 06:41:57,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:57,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ee291f903fec0b0e086b HTTP/1.1" 200 None 2026-05-05 06:41:57,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:57,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:41:57,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:41:57,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ee291f903fec0b0e086b HTTP/1.1" 200 None 2026-05-05 06:41:57,922 - lava_callback - ERROR - Error processing callback for node 69f6ee291f903fec0b0e086b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36752 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:43:59,112 - lava_callback - INFO - Received callback for node 69f6ee991f903fec0b0e0895 INFO: 172.20.0.1:36764 - "POST /node/69f6ee991f903fec0b0e0895 HTTP/1.0" 202 Accepted 2026-05-05 06:43:59,118 - lava_callback - INFO - Starting async job submit for node 69f6ee991f903fec0b0e0895 2026-05-05 06:43:59,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:43:59,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ee991f903fec0b0e0895 HTTP/1.1" 200 None 2026-05-05 06:43:59,133 - lava_callback - DEBUG - Processing node 69f6ee991f903fec0b0e0895: baseline-arm-clabbe 2026-05-05 06:43:59,134 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ee991f903fec0b0e0895 2026-05-05 06:43:59,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:43:59,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ee991f903fec0b0e0895 HTTP/1.1" 200 None 2026-05-05 06:43:59,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:43:59,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f380a62fa6ff8e8ca8e094 HTTP/1.1" 200 None 2026-05-05 06:43:59,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:43:59,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ee991f903fec0b0e0895 HTTP/1.1" 200 None 2026-05-05 06:43:59,204 - lava_callback - ERROR - Error processing callback for node 69f6ee991f903fec0b0e0895: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48990 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:45:25,217 - lava_callback - INFO - Received callback for node 69f6eeff1f903fec0b0e08b9 INFO: 172.20.0.1:41768 - "POST /node/69f6eeff1f903fec0b0e08b9 HTTP/1.0" 202 Accepted 2026-05-05 06:45:25,223 - lava_callback - INFO - Starting async job submit for node 69f6eeff1f903fec0b0e08b9 2026-05-05 06:45:25,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:45:25,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eeff1f903fec0b0e08b9 HTTP/1.1" 200 None 2026-05-05 06:45:25,251 - lava_callback - DEBUG - Processing node 69f6eeff1f903fec0b0e08b9: baseline-arm-baylibre 2026-05-05 06:45:25,251 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6eeff1f903fec0b0e08b9 2026-05-05 06:45:25,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:45:25,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6eeff1f903fec0b0e08b9 HTTP/1.1" 200 None 2026-05-05 06:45:25,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:45:25,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 06:45:25,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:45:25,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6eeff1f903fec0b0e08b9 HTTP/1.1" 200 None 2026-05-05 06:45:25,307 - lava_callback - ERROR - Error processing callback for node 69f6eeff1f903fec0b0e08b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37302 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:47:18,165 - lava_callback - INFO - Received callback for node 69f6ef6f1f903fec0b0e08e5 INFO: 172.20.0.1:37314 - "POST /node/69f6ef6f1f903fec0b0e08e5 HTTP/1.0" 202 Accepted 2026-05-05 06:47:18,171 - lava_callback - INFO - Starting async job submit for node 69f6ef6f1f903fec0b0e08e5 2026-05-05 06:47:18,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:47:18,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ef6f1f903fec0b0e08e5 HTTP/1.1" 200 None 2026-05-05 06:47:18,190 - lava_callback - DEBUG - Processing node 69f6ef6f1f903fec0b0e08e5: baseline-arm-clabbe 2026-05-05 06:47:18,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ef6f1f903fec0b0e08e5 2026-05-05 06:47:18,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:47:18,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ef6f1f903fec0b0e08e5 HTTP/1.1" 200 None 2026-05-05 06:47:18,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:47:18,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 06:47:18,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:47:18,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ef6f1f903fec0b0e08e5 HTTP/1.1" 200 None 2026-05-05 06:47:18,237 - lava_callback - ERROR - Error processing callback for node 69f6ef6f1f903fec0b0e08e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49488 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:49:18,141 - lava_callback - INFO - Received callback for node 69f6efde1f903fec0b0e0907 INFO: 172.20.0.1:49502 - "POST /node/69f6efde1f903fec0b0e0907 HTTP/1.0" 202 Accepted 2026-05-05 06:49:18,147 - lava_callback - INFO - Starting async job submit for node 69f6efde1f903fec0b0e0907 2026-05-05 06:49:18,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:18,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6efde1f903fec0b0e0907 HTTP/1.1" 200 None 2026-05-05 06:49:18,168 - lava_callback - DEBUG - Processing node 69f6efde1f903fec0b0e0907: baseline-arm-clabbe 2026-05-05 06:49:18,169 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6efde1f903fec0b0e0907 2026-05-05 06:49:18,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:18,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6efde1f903fec0b0e0907 HTTP/1.1" 200 None 2026-05-05 06:49:18,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:18,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 06:49:18,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:18,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6efde1f903fec0b0e0907 HTTP/1.1" 200 None 2026-05-05 06:49:18,213 - lava_callback - ERROR - Error processing callback for node 69f6efde1f903fec0b0e0907: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48696 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:49:58,150 - lava_callback - INFO - Received callback for node 69f6f0041f903fec0b0e0916 INFO: 172.20.0.1:48712 - "POST /node/69f6f0041f903fec0b0e0916 HTTP/1.0" 202 Accepted 2026-05-05 06:49:58,155 - lava_callback - INFO - Starting async job submit for node 69f6f0041f903fec0b0e0916 2026-05-05 06:49:58,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:58,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0041f903fec0b0e0916 HTTP/1.1" 200 None 2026-05-05 06:49:58,170 - lava_callback - DEBUG - Processing node 69f6f0041f903fec0b0e0916: baseline-arm-clabbe 2026-05-05 06:49:58,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f0041f903fec0b0e0916 2026-05-05 06:49:58,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:58,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0041f903fec0b0e0916 HTTP/1.1" 200 None 2026-05-05 06:49:58,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:58,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 06:49:58,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:49:58,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f0041f903fec0b0e0916 HTTP/1.1" 200 None 2026-05-05 06:49:58,216 - lava_callback - ERROR - Error processing callback for node 69f6f0041f903fec0b0e0916: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43090 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:50:38,243 - lava_callback - INFO - Received callback for node 69f6f0291f903fec0b0e0924 INFO: 172.20.0.1:43104 - "POST /node/69f6f0291f903fec0b0e0924 HTTP/1.0" 202 Accepted 2026-05-05 06:50:38,252 - lava_callback - INFO - Starting async job submit for node 69f6f0291f903fec0b0e0924 2026-05-05 06:50:38,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:50:38,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0291f903fec0b0e0924 HTTP/1.1" 200 None 2026-05-05 06:50:38,278 - lava_callback - DEBUG - Processing node 69f6f0291f903fec0b0e0924: baseline-arm-clabbe 2026-05-05 06:50:38,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f0291f903fec0b0e0924 2026-05-05 06:50:38,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:50:38,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0291f903fec0b0e0924 HTTP/1.1" 200 None 2026-05-05 06:50:38,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:50:38,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 06:50:38,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:50:38,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f0291f903fec0b0e0924 HTTP/1.1" 200 None 2026-05-05 06:50:38,330 - lava_callback - ERROR - Error processing callback for node 69f6f0291f903fec0b0e0924: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:52:18,260 - lava_callback - INFO - Received callback for node 69f6f0971f903fec0b0e094d INFO: 172.20.0.1:59170 - "POST /node/69f6f0971f903fec0b0e094d HTTP/1.0" 202 Accepted 2026-05-05 06:52:18,266 - lava_callback - INFO - Starting async job submit for node 69f6f0971f903fec0b0e094d 2026-05-05 06:52:18,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:18,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0971f903fec0b0e094d HTTP/1.1" 200 None 2026-05-05 06:52:18,284 - lava_callback - DEBUG - Processing node 69f6f0971f903fec0b0e094d: baseline-arm-clabbe 2026-05-05 06:52:18,284 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f0971f903fec0b0e094d 2026-05-05 06:52:18,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:18,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0971f903fec0b0e094d HTTP/1.1" 200 None 2026-05-05 06:52:18,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:18,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6942d1f903fec0b0dbc77 HTTP/1.1" 200 None 2026-05-05 06:52:18,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:18,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f0971f903fec0b0e094d HTTP/1.1" 200 None 2026-05-05 06:52:18,350 - lava_callback - ERROR - Error processing callback for node 69f6f0971f903fec0b0e094d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45642 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:52:45,651 - lava_callback - INFO - Received callback for node 69f6f0b61f903fec0b0e0959 INFO: 172.20.0.1:45518 - "POST /node/69f6f0b61f903fec0b0e0959 HTTP/1.0" 202 Accepted 2026-05-05 06:52:45,656 - lava_callback - INFO - Starting async job submit for node 69f6f0b61f903fec0b0e0959 2026-05-05 06:52:45,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:45,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0b61f903fec0b0e0959 HTTP/1.1" 200 None 2026-05-05 06:52:45,671 - lava_callback - DEBUG - Processing node 69f6f0b61f903fec0b0e0959: baseline-arm-baylibre 2026-05-05 06:52:45,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f0b61f903fec0b0e0959 2026-05-05 06:52:45,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:45,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f0b61f903fec0b0e0959 HTTP/1.1" 200 None 2026-05-05 06:52:45,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:45,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:52:45,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:52:45,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f0b61f903fec0b0e0959 HTTP/1.1" 200 None 2026-05-05 06:52:45,721 - lava_callback - ERROR - Error processing callback for node 69f6f0b61f903fec0b0e0959: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34744 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:54:25,626 - lava_callback - INFO - Received callback for node 69f6f1181f903fec0b0e0991 INFO: 172.20.0.1:54720 - "POST /node/69f6f1181f903fec0b0e0991 HTTP/1.0" 202 Accepted 2026-05-05 06:54:25,631 - lava_callback - INFO - Starting async job submit for node 69f6f1181f903fec0b0e0991 2026-05-05 06:54:25,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:54:25,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1181f903fec0b0e0991 HTTP/1.1" 200 None 2026-05-05 06:54:25,646 - lava_callback - DEBUG - Processing node 69f6f1181f903fec0b0e0991: baseline-arm-baylibre 2026-05-05 06:54:25,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f1181f903fec0b0e0991 2026-05-05 06:54:25,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:54:25,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1181f903fec0b0e0991 HTTP/1.1" 200 None 2026-05-05 06:54:25,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:54:25,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:54:25,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:54:25,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f1181f903fec0b0e0991 HTTP/1.1" 200 None 2026-05-05 06:54:25,699 - lava_callback - ERROR - Error processing callback for node 69f6f1181f903fec0b0e0991: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:55:05,654 - lava_callback - INFO - Received callback for node 69f6f13d1f903fec0b0e09ac INFO: 172.20.0.1:58066 - "POST /node/69f6f13d1f903fec0b0e09ac HTTP/1.0" 202 Accepted 2026-05-05 06:55:05,659 - lava_callback - INFO - Starting async job submit for node 69f6f13d1f903fec0b0e09ac 2026-05-05 06:55:05,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:05,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f13d1f903fec0b0e09ac HTTP/1.1" 200 None 2026-05-05 06:55:05,674 - lava_callback - DEBUG - Processing node 69f6f13d1f903fec0b0e09ac: baseline-arm-baylibre 2026-05-05 06:55:05,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f13d1f903fec0b0e09ac 2026-05-05 06:55:05,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:05,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f13d1f903fec0b0e09ac HTTP/1.1" 200 None 2026-05-05 06:55:05,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:05,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:55:05,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:05,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f13d1f903fec0b0e09ac HTTP/1.1" 200 None 2026-05-05 06:55:05,727 - lava_callback - ERROR - Error processing callback for node 69f6f13d1f903fec0b0e09ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46172 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:55:45,753 - lava_callback - INFO - Received callback for node 69f6f1611f903fec0b0e09bc INFO: 172.20.0.1:58726 - "POST /node/69f6f1611f903fec0b0e09bc HTTP/1.0" 202 Accepted 2026-05-05 06:55:45,759 - lava_callback - INFO - Starting async job submit for node 69f6f1611f903fec0b0e09bc 2026-05-05 06:55:45,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:45,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1611f903fec0b0e09bc HTTP/1.1" 200 None 2026-05-05 06:55:45,776 - lava_callback - DEBUG - Processing node 69f6f1611f903fec0b0e09bc: baseline-arm-baylibre 2026-05-05 06:55:45,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f1611f903fec0b0e09bc 2026-05-05 06:55:45,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:45,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1611f903fec0b0e09bc HTTP/1.1" 200 None 2026-05-05 06:55:45,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:45,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:55:45,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:55:45,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f1611f903fec0b0e09bc HTTP/1.1" 200 None 2026-05-05 06:55:45,837 - lava_callback - ERROR - Error processing callback for node 69f6f1611f903fec0b0e09bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60802 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:56:25,721 - lava_callback - INFO - Received callback for node 69f6f1851f903fec0b0e09ca INFO: 172.20.0.1:43504 - "POST /node/69f6f1851f903fec0b0e09ca HTTP/1.0" 202 Accepted 2026-05-05 06:56:25,727 - lava_callback - INFO - Starting async job submit for node 69f6f1851f903fec0b0e09ca 2026-05-05 06:56:25,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:25,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1851f903fec0b0e09ca HTTP/1.1" 200 None 2026-05-05 06:56:25,742 - lava_callback - DEBUG - Processing node 69f6f1851f903fec0b0e09ca: baseline-arm-baylibre 2026-05-05 06:56:25,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f1851f903fec0b0e09ca 2026-05-05 06:56:25,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:25,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1851f903fec0b0e09ca HTTP/1.1" 200 None 2026-05-05 06:56:25,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:25,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:56:25,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:25,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f1851f903fec0b0e09ca HTTP/1.1" 200 None 2026-05-05 06:56:25,832 - lava_callback - ERROR - Error processing callback for node 69f6f1851f903fec0b0e09ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:56:45,757 - lava_callback - INFO - Received callback for node 69f6f1a61f903fec0b0e09de INFO: 172.20.0.1:50786 - "POST /node/69f6f1a61f903fec0b0e09de HTTP/1.0" 202 Accepted 2026-05-05 06:56:45,764 - lava_callback - INFO - Starting async job submit for node 69f6f1a61f903fec0b0e09de 2026-05-05 06:56:45,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:45,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1a61f903fec0b0e09de HTTP/1.1" 200 None 2026-05-05 06:56:45,781 - lava_callback - DEBUG - Processing node 69f6f1a61f903fec0b0e09de: baseline-arm-baylibre 2026-05-05 06:56:45,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f1a61f903fec0b0e09de 2026-05-05 06:56:45,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:45,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1a61f903fec0b0e09de HTTP/1.1" 200 None 2026-05-05 06:56:45,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:45,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:56:45,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:56:45,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f1a61f903fec0b0e09de HTTP/1.1" 200 None 2026-05-05 06:56:45,841 - lava_callback - ERROR - Error processing callback for node 69f6f1a61f903fec0b0e09de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49722 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:57:38,521 - lava_callback - INFO - Received callback for node 69f6f1ce1f903fec0b0e09eb INFO: 172.20.0.1:49730 - "POST /node/69f6f1ce1f903fec0b0e09eb HTTP/1.0" 202 Accepted 2026-05-05 06:57:38,527 - lava_callback - INFO - Starting async job submit for node 69f6f1ce1f903fec0b0e09eb 2026-05-05 06:57:38,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:57:38,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1ce1f903fec0b0e09eb HTTP/1.1" 200 None 2026-05-05 06:57:38,542 - lava_callback - DEBUG - Processing node 69f6f1ce1f903fec0b0e09eb: baseline-arm-clabbe 2026-05-05 06:57:38,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f1ce1f903fec0b0e09eb 2026-05-05 06:57:38,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:57:38,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f1ce1f903fec0b0e09eb HTTP/1.1" 200 None 2026-05-05 06:57:38,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:57:38,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:57:38,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:57:38,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f1ce1f903fec0b0e09eb HTTP/1.1" 200 None 2026-05-05 06:57:38,594 - lava_callback - ERROR - Error processing callback for node 69f6f1ce1f903fec0b0e09eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45056 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:58:45,992 - lava_callback - INFO - Received callback for node 69f8d1e40573a8210e1300c6 INFO: 172.20.0.1:59166 - "POST /node/69f8d1e40573a8210e1300c6 HTTP/1.0" 202 Accepted 2026-05-05 06:58:45,998 - lava_callback - INFO - Starting async job submit for node 69f8d1e40573a8210e1300c6 2026-05-05 06:58:46,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:58:46,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f8d1e40573a8210e1300c6 HTTP/1.1" 200 None 2026-05-05 06:58:46,036 - lava_callback - DEBUG - Processing node 69f8d1e40573a8210e1300c6: baseline-arm64-broonie 2026-05-05 06:58:46,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 06:58:46,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 06:58:46,442 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f8d1e40573a8210e1300c6/log.txt.gz INFO: 172.20.0.1:59172 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:58:46,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 06:58:46,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 06:58:46,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f8d1e40573a8210e1300c6/lava-logs.yaml 2026-05-05 06:58:46,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 06:58:47,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 06:58:47,010 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f8d1e40573a8210e1300c6/lava_callback.json.gz 2026-05-05 06:58:47,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:58:47,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f8d1e40573a8210e1300c6 HTTP/1.1" 200 None 2026-05-05 06:58:47,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:58:47,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f8c6760573a8210e12ccd7 HTTP/1.1" 200 None 2026-05-05 06:58:47,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:58:47,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f8d1e40573a8210e1300c6 HTTP/1.1" 200 None 2026-05-05 06:58:47,114 - lava_callback - INFO - Completed processing callback for node 69f8d1e40573a8210e1300c6 INFO: 172.20.0.1:52928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:59:15,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:15,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47600 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:59:18,575 - lava_callback - INFO - Received callback for node 69f6f23d1f903fec0b0e0a20 INFO: 172.20.0.1:47602 - "POST /node/69f6f23d1f903fec0b0e0a20 HTTP/1.0" 202 Accepted 2026-05-05 06:59:18,581 - lava_callback - INFO - Starting async job submit for node 69f6f23d1f903fec0b0e0a20 2026-05-05 06:59:18,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:18,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f23d1f903fec0b0e0a20 HTTP/1.1" 200 None 2026-05-05 06:59:18,599 - lava_callback - DEBUG - Processing node 69f6f23d1f903fec0b0e0a20: baseline-arm-clabbe 2026-05-05 06:59:18,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f23d1f903fec0b0e0a20 2026-05-05 06:59:18,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:18,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f23d1f903fec0b0e0a20 HTTP/1.1" 200 None 2026-05-05 06:59:18,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:18,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:59:18,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:18,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f23d1f903fec0b0e0a20 HTTP/1.1" 200 None 2026-05-05 06:59:18,649 - lava_callback - ERROR - Error processing callback for node 69f6f23d1f903fec0b0e0a20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43346 - "GET / HTTP/1.0" 200 OK 2026-05-05 06:59:58,535 - lava_callback - INFO - Received callback for node 69f6f2621f903fec0b0e0a2b INFO: 172.20.0.1:43360 - "POST /node/69f6f2621f903fec0b0e0a2b HTTP/1.0" 202 Accepted 2026-05-05 06:59:58,541 - lava_callback - INFO - Starting async job submit for node 69f6f2621f903fec0b0e0a2b 2026-05-05 06:59:58,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:58,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f2621f903fec0b0e0a2b HTTP/1.1" 200 None 2026-05-05 06:59:58,556 - lava_callback - DEBUG - Processing node 69f6f2621f903fec0b0e0a2b: baseline-arm-clabbe 2026-05-05 06:59:58,557 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f2621f903fec0b0e0a2b 2026-05-05 06:59:58,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:58,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f2621f903fec0b0e0a2b HTTP/1.1" 200 None 2026-05-05 06:59:58,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:58,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 06:59:58,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 06:59:58,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f2621f903fec0b0e0a2b HTTP/1.1" 200 None 2026-05-05 06:59:58,620 - lava_callback - ERROR - Error processing callback for node 69f6f2621f903fec0b0e0a2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33094 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:00:38,862 - lava_callback - INFO - Received callback for node 69f6f2871f903fec0b0e0a3b INFO: 172.20.0.1:33104 - "POST /node/69f6f2871f903fec0b0e0a3b HTTP/1.0" 202 Accepted 2026-05-05 07:00:38,868 - lava_callback - INFO - Starting async job submit for node 69f6f2871f903fec0b0e0a3b 2026-05-05 07:00:38,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:38,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f2871f903fec0b0e0a3b HTTP/1.1" 200 None 2026-05-05 07:00:38,907 - lava_callback - DEBUG - Processing node 69f6f2871f903fec0b0e0a3b: baseline-arm-clabbe 2026-05-05 07:00:38,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f2871f903fec0b0e0a3b 2026-05-05 07:00:38,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:38,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f2871f903fec0b0e0a3b HTTP/1.1" 200 None 2026-05-05 07:00:38,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:38,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:00:38,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:38,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f2871f903fec0b0e0a3b HTTP/1.1" 200 None 2026-05-05 07:00:38,953 - lava_callback - ERROR - Error processing callback for node 69f6f2871f903fec0b0e0a3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-05-05 07:00:46,074 - lava_callback - INFO - Received callback for node 69f92d3a0573a8210e132fd1 INFO: 172.20.0.1:36170 - "POST /node/69f92d3a0573a8210e132fd1 HTTP/1.0" 202 Accepted 2026-05-05 07:00:46,080 - lava_callback - INFO - Starting async job submit for node 69f92d3a0573a8210e132fd1 2026-05-05 07:00:46,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:46,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f92d3a0573a8210e132fd1 HTTP/1.1" 200 None 2026-05-05 07:00:46,117 - lava_callback - DEBUG - Processing node 69f92d3a0573a8210e132fd1: baseline-arm64-broonie 2026-05-05 07:00:46,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:00:46,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:00:46,510 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f92d3a0573a8210e132fd1/log.txt.gz INFO: 172.20.0.1:36184 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:00:46,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:00:46,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:00:46,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f92d3a0573a8210e132fd1/lava-logs.yaml 2026-05-05 07:00:46,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:00:47,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:00:47,026 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f92d3a0573a8210e132fd1/lava_callback.json.gz 2026-05-05 07:00:47,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:47,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f92d3a0573a8210e132fd1 HTTP/1.1" 200 None 2026-05-05 07:00:47,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:47,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f8c6760573a8210e12ccd7 HTTP/1.1" 200 None 2026-05-05 07:00:47,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:00:47,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f92d3a0573a8210e132fd1 HTTP/1.1" 200 None 2026-05-05 07:00:47,121 - lava_callback - INFO - Completed processing callback for node 69f92d3a0573a8210e132fd1 INFO: 172.20.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60392 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:01:15,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:01:15,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45848 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:01:18,607 - lava_callback - INFO - Received callback for node 69f6f2ac1f903fec0b0e0a4a INFO: 172.20.0.1:45862 - "POST /node/69f6f2ac1f903fec0b0e0a4a HTTP/1.0" 202 Accepted 2026-05-05 07:01:18,612 - lava_callback - INFO - Starting async job submit for node 69f6f2ac1f903fec0b0e0a4a 2026-05-05 07:01:18,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:01:18,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f2ac1f903fec0b0e0a4a HTTP/1.1" 200 None 2026-05-05 07:01:18,628 - lava_callback - DEBUG - Processing node 69f6f2ac1f903fec0b0e0a4a: baseline-arm-clabbe 2026-05-05 07:01:18,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f2ac1f903fec0b0e0a4a 2026-05-05 07:01:18,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:01:18,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f2ac1f903fec0b0e0a4a HTTP/1.1" 200 None 2026-05-05 07:01:18,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:01:18,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:01:18,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:01:18,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f2ac1f903fec0b0e0a4a HTTP/1.1" 200 None 2026-05-05 07:01:18,685 - lava_callback - ERROR - Error processing callback for node 69f6f2ac1f903fec0b0e0a4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:02:47,047 - lava_callback - INFO - Received callback for node 69f9428dd09fe59064f05a51 INFO: 172.20.0.1:46796 - "POST /node/69f9428dd09fe59064f05a51 HTTP/1.0" 202 Accepted 2026-05-05 07:02:47,053 - lava_callback - INFO - Starting async job submit for node 69f9428dd09fe59064f05a51 2026-05-05 07:02:47,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:02:47,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9428dd09fe59064f05a51 HTTP/1.1" 200 None 2026-05-05 07:02:47,094 - lava_callback - DEBUG - Processing node 69f9428dd09fe59064f05a51: baseline-arm64-broonie 2026-05-05 07:02:47,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:02:47,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:02:47,547 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428dd09fe59064f05a51/log.txt.gz 2026-05-05 07:02:47,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:02:47,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:02:47,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428dd09fe59064f05a51/lava-logs.yaml 2026-05-05 07:02:47,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:02:48,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:02:48,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9428dd09fe59064f05a51/lava_callback.json.gz 2026-05-05 07:02:48,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:02:48,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9428dd09fe59064f05a51 HTTP/1.1" 200 None 2026-05-05 07:02:48,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:02:48,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f93713d09fe59064f02d00 HTTP/1.1" 200 None 2026-05-05 07:02:48,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:02:48,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9428dd09fe59064f05a51 HTTP/1.1" 200 None 2026-05-05 07:02:48,103 - lava_callback - INFO - Completed processing callback for node 69f9428dd09fe59064f05a51 INFO: 172.20.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33222 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:03:15,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:03:15,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:04:46,238 - lava_callback - INFO - Received callback for node 69f9844ad09fe59064f0737e INFO: 172.20.0.1:52706 - "POST /node/69f9844ad09fe59064f0737e HTTP/1.0" 202 Accepted 2026-05-05 07:04:46,244 - lava_callback - INFO - Starting async job submit for node 69f9844ad09fe59064f0737e 2026-05-05 07:04:46,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:04:46,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9844ad09fe59064f0737e HTTP/1.1" 200 None 2026-05-05 07:04:46,281 - lava_callback - DEBUG - Processing node 69f9844ad09fe59064f0737e: baseline-arm64-broonie INFO: 172.20.0.1:52718 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:04:46,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:04:46,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:04:46,677 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9844ad09fe59064f0737e/log.txt.gz 2026-05-05 07:04:46,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:04:47,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:04:47,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9844ad09fe59064f0737e/lava-logs.yaml 2026-05-05 07:04:47,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-05-05 07:04:47,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-05-05 07:04:47,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69f9844ad09fe59064f0737e/lava_callback.json.gz 2026-05-05 07:04:47,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:04:47,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f9844ad09fe59064f0737e HTTP/1.1" 200 None 2026-05-05 07:04:47,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:04:47,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f8c6760573a8210e12ccd7 HTTP/1.1" 200 None 2026-05-05 07:04:47,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:04:47,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f9844ad09fe59064f0737e HTTP/1.1" 200 None 2026-05-05 07:04:47,296 - lava_callback - INFO - Completed processing callback for node 69f9844ad09fe59064f0737e INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44190 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:05:15,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:39656 - "GET /api/metrics HTTP/1.0" 200 OK 2026-05-05 07:05:15,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53874 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:06:46,633 - lava_callback - INFO - Received callback for node 69f6f3f31f903fec0b0e0af4 INFO: 172.20.0.1:53880 - "POST /node/69f6f3f31f903fec0b0e0af4 HTTP/1.0" 202 Accepted 2026-05-05 07:06:46,639 - lava_callback - INFO - Starting async job submit for node 69f6f3f31f903fec0b0e0af4 2026-05-05 07:06:46,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:06:46,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f3f31f903fec0b0e0af4 HTTP/1.1" 200 None 2026-05-05 07:06:46,654 - lava_callback - DEBUG - Processing node 69f6f3f31f903fec0b0e0af4: baseline-arm-baylibre 2026-05-05 07:06:46,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f3f31f903fec0b0e0af4 2026-05-05 07:06:46,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:06:46,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f3f31f903fec0b0e0af4 HTTP/1.1" 200 None 2026-05-05 07:06:46,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:06:46,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:06:46,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:06:46,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f3f31f903fec0b0e0af4 HTTP/1.1" 200 None 2026-05-05 07:06:46,702 - lava_callback - ERROR - Error processing callback for node 69f6f3f31f903fec0b0e0af4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39446 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:07:26,302 - lava_callback - INFO - Received callback for node 69f6f4231f903fec0b0e0b0b INFO: 172.20.0.1:32812 - "POST /node/69f6f4231f903fec0b0e0b0b HTTP/1.0" 202 Accepted 2026-05-05 07:07:26,308 - lava_callback - INFO - Starting async job submit for node 69f6f4231f903fec0b0e0b0b 2026-05-05 07:07:26,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:07:26,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f4231f903fec0b0e0b0b HTTP/1.1" 200 None 2026-05-05 07:07:26,347 - lava_callback - DEBUG - Processing node 69f6f4231f903fec0b0e0b0b: baseline-arm64-baylibre 2026-05-05 07:07:26,347 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f4231f903fec0b0e0b0b 2026-05-05 07:07:26,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:07:26,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f4231f903fec0b0e0b0b HTTP/1.1" 200 None 2026-05-05 07:07:26,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:07:26,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 07:07:26,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:07:26,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f4231f903fec0b0e0b0b HTTP/1.1" 200 None 2026-05-05 07:07:26,394 - lava_callback - ERROR - Error processing callback for node 69f6f4231f903fec0b0e0b0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58634 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:08:26,392 - lava_callback - INFO - Received callback for node 69f6f4601f903fec0b0e0b2c INFO: 172.20.0.1:52276 - "POST /node/69f6f4601f903fec0b0e0b2c HTTP/1.0" 202 Accepted 2026-05-05 07:08:26,397 - lava_callback - INFO - Starting async job submit for node 69f6f4601f903fec0b0e0b2c 2026-05-05 07:08:26,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:08:26,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f4601f903fec0b0e0b2c HTTP/1.1" 200 None 2026-05-05 07:08:26,412 - lava_callback - DEBUG - Processing node 69f6f4601f903fec0b0e0b2c: baseline-arm-baylibre 2026-05-05 07:08:26,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f4601f903fec0b0e0b2c 2026-05-05 07:08:26,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:08:26,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f4601f903fec0b0e0b2c HTTP/1.1" 200 None 2026-05-05 07:08:26,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:08:26,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:08:26,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:08:26,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f4601f903fec0b0e0b2c HTTP/1.1" 200 None 2026-05-05 07:08:26,460 - lava_callback - ERROR - Error processing callback for node 69f6f4601f903fec0b0e0b2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37818 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:10:39,036 - lava_callback - INFO - Received callback for node 69f6f4df1f903fec0b0e0b60 INFO: 172.20.0.1:37820 - "POST /node/69f6f4df1f903fec0b0e0b60 HTTP/1.0" 202 Accepted 2026-05-05 07:10:39,042 - lava_callback - INFO - Starting async job submit for node 69f6f4df1f903fec0b0e0b60 2026-05-05 07:10:39,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:10:39,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f4df1f903fec0b0e0b60 HTTP/1.1" 200 None 2026-05-05 07:10:39,058 - lava_callback - DEBUG - Processing node 69f6f4df1f903fec0b0e0b60: baseline-arm64-clabbe 2026-05-05 07:10:39,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f4df1f903fec0b0e0b60 2026-05-05 07:10:39,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:10:39,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f4df1f903fec0b0e0b60 HTTP/1.1" 200 None 2026-05-05 07:10:39,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:10:39,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 07:10:39,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:10:39,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f4df1f903fec0b0e0b60 HTTP/1.1" 200 None 2026-05-05 07:10:39,115 - lava_callback - ERROR - Error processing callback for node 69f6f4df1f903fec0b0e0b60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:11:59,031 - lava_callback - INFO - Received callback for node 69f6f5291f903fec0b0e0b94 INFO: 172.20.0.1:36636 - "POST /node/69f6f5291f903fec0b0e0b94 HTTP/1.0" 202 Accepted 2026-05-05 07:11:59,040 - lava_callback - INFO - Starting async job submit for node 69f6f5291f903fec0b0e0b94 2026-05-05 07:11:59,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:11:59,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f5291f903fec0b0e0b94 HTTP/1.1" 200 None 2026-05-05 07:11:59,056 - lava_callback - DEBUG - Processing node 69f6f5291f903fec0b0e0b94: baseline-arm64-clabbe 2026-05-05 07:11:59,056 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f5291f903fec0b0e0b94 2026-05-05 07:11:59,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:11:59,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f5291f903fec0b0e0b94 HTTP/1.1" 200 None 2026-05-05 07:11:59,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:11:59,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 07:11:59,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:11:59,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f5291f903fec0b0e0b94 HTTP/1.1" 200 None 2026-05-05 07:11:59,119 - lava_callback - ERROR - Error processing callback for node 69f6f5291f903fec0b0e0b94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53862 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:12:59,151 - lava_callback - INFO - Received callback for node 69f6f5731f903fec0b0e0bb2 INFO: 172.20.0.1:53866 - "POST /node/69f6f5731f903fec0b0e0bb2 HTTP/1.0" 202 Accepted 2026-05-05 07:12:59,157 - lava_callback - INFO - Starting async job submit for node 69f6f5731f903fec0b0e0bb2 2026-05-05 07:12:59,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:12:59,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f5731f903fec0b0e0bb2 HTTP/1.1" 200 None 2026-05-05 07:12:59,172 - lava_callback - DEBUG - Processing node 69f6f5731f903fec0b0e0bb2: baseline-arm64-clabbe 2026-05-05 07:12:59,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f5731f903fec0b0e0bb2 2026-05-05 07:12:59,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:12:59,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f5731f903fec0b0e0bb2 HTTP/1.1" 200 None 2026-05-05 07:12:59,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:12:59,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 07:12:59,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:12:59,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f5731f903fec0b0e0bb2 HTTP/1.1" 200 None 2026-05-05 07:12:59,235 - lava_callback - ERROR - Error processing callback for node 69f6f5731f903fec0b0e0bb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54260 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:14:19,196 - lava_callback - INFO - Received callback for node 69f6f5bc1f903fec0b0e0bd5 INFO: 172.20.0.1:54268 - "POST /node/69f6f5bc1f903fec0b0e0bd5 HTTP/1.0" 202 Accepted 2026-05-05 07:14:19,201 - lava_callback - INFO - Starting async job submit for node 69f6f5bc1f903fec0b0e0bd5 2026-05-05 07:14:19,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:14:19,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f5bc1f903fec0b0e0bd5 HTTP/1.1" 200 None 2026-05-05 07:14:19,218 - lava_callback - DEBUG - Processing node 69f6f5bc1f903fec0b0e0bd5: baseline-arm64-clabbe 2026-05-05 07:14:19,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f5bc1f903fec0b0e0bd5 2026-05-05 07:14:19,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:14:19,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f5bc1f903fec0b0e0bd5 HTTP/1.1" 200 None 2026-05-05 07:14:19,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:14:19,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 07:14:19,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:14:19,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f5bc1f903fec0b0e0bd5 HTTP/1.1" 200 None 2026-05-05 07:14:19,281 - lava_callback - ERROR - Error processing callback for node 69f6f5bc1f903fec0b0e0bd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:16:39,188 - lava_callback - INFO - Received callback for node 69f6f6541f903fec0b0e0c1d INFO: 172.20.0.1:52406 - "POST /node/69f6f6541f903fec0b0e0c1d HTTP/1.0" 202 Accepted 2026-05-05 07:16:39,194 - lava_callback - INFO - Starting async job submit for node 69f6f6541f903fec0b0e0c1d 2026-05-05 07:16:39,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:16:39,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f6541f903fec0b0e0c1d HTTP/1.1" 200 None 2026-05-05 07:16:39,209 - lava_callback - DEBUG - Processing node 69f6f6541f903fec0b0e0c1d: baseline-arm64-clabbe 2026-05-05 07:16:39,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f6541f903fec0b0e0c1d 2026-05-05 07:16:39,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:16:39,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f6541f903fec0b0e0c1d HTTP/1.1" 200 None 2026-05-05 07:16:39,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:16:39,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 07:16:39,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:16:39,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f6541f903fec0b0e0c1d HTTP/1.1" 200 None 2026-05-05 07:16:39,264 - lava_callback - ERROR - Error processing callback for node 69f6f6541f903fec0b0e0c1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60392 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:17:59,304 - lava_callback - INFO - Received callback for node 69f6f69c1f903fec0b0e0c4d INFO: 172.20.0.1:60396 - "POST /node/69f6f69c1f903fec0b0e0c4d HTTP/1.0" 202 Accepted 2026-05-05 07:17:59,310 - lava_callback - INFO - Starting async job submit for node 69f6f69c1f903fec0b0e0c4d 2026-05-05 07:17:59,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:17:59,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f69c1f903fec0b0e0c4d HTTP/1.1" 200 None 2026-05-05 07:17:59,325 - lava_callback - DEBUG - Processing node 69f6f69c1f903fec0b0e0c4d: baseline-arm64-clabbe 2026-05-05 07:17:59,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f69c1f903fec0b0e0c4d 2026-05-05 07:17:59,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:17:59,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f69c1f903fec0b0e0c4d HTTP/1.1" 200 None 2026-05-05 07:17:59,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:17:59,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6943a1f903fec0b0dbc8b HTTP/1.1" 200 None 2026-05-05 07:17:59,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:17:59,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f69c1f903fec0b0e0c4d HTTP/1.1" 200 None 2026-05-05 07:17:59,386 - lava_callback - ERROR - Error processing callback for node 69f6f69c1f903fec0b0e0c4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41962 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:19:27,066 - lava_callback - INFO - Received callback for node 69f6f6fa1f903fec0b0e0c7b INFO: 172.20.0.1:41970 - "POST /node/69f6f6fa1f903fec0b0e0c7b HTTP/1.0" 202 Accepted 2026-05-05 07:19:27,071 - lava_callback - INFO - Starting async job submit for node 69f6f6fa1f903fec0b0e0c7b 2026-05-05 07:19:27,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:19:27,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f6fa1f903fec0b0e0c7b HTTP/1.1" 200 None 2026-05-05 07:19:27,086 - lava_callback - DEBUG - Processing node 69f6f6fa1f903fec0b0e0c7b: baseline-arm-baylibre 2026-05-05 07:19:27,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f6fa1f903fec0b0e0c7b 2026-05-05 07:19:27,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:19:27,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f6fa1f903fec0b0e0c7b HTTP/1.1" 200 None 2026-05-05 07:19:27,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:19:27,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:19:27,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:19:27,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f6fa1f903fec0b0e0c7b HTTP/1.1" 200 None 2026-05-05 07:19:27,140 - lava_callback - ERROR - Error processing callback for node 69f6f6fa1f903fec0b0e0c7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35952 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:20:59,456 - lava_callback - INFO - Received callback for node 69f6f7451f903fec0b0e0c9d INFO: 172.20.0.1:35958 - "POST /node/69f6f7451f903fec0b0e0c9d HTTP/1.0" 202 Accepted 2026-05-05 07:20:59,462 - lava_callback - INFO - Starting async job submit for node 69f6f7451f903fec0b0e0c9d 2026-05-05 07:20:59,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:20:59,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7451f903fec0b0e0c9d HTTP/1.1" 200 None 2026-05-05 07:20:59,481 - lava_callback - DEBUG - Processing node 69f6f7451f903fec0b0e0c9d: baseline-arm-clabbe 2026-05-05 07:20:59,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f7451f903fec0b0e0c9d 2026-05-05 07:20:59,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:20:59,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7451f903fec0b0e0c9d HTTP/1.1" 200 None 2026-05-05 07:20:59,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:20:59,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:20:59,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:20:59,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f7451f903fec0b0e0c9d HTTP/1.1" 200 None 2026-05-05 07:20:59,542 - lava_callback - ERROR - Error processing callback for node 69f6f7451f903fec0b0e0c9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:21:59,536 - lava_callback - INFO - Received callback for node 69f6f78c1f903fec0b0e0cbb INFO: 172.20.0.1:42438 - "POST /node/69f6f78c1f903fec0b0e0cbb HTTP/1.0" 202 Accepted 2026-05-05 07:21:59,541 - lava_callback - INFO - Starting async job submit for node 69f6f78c1f903fec0b0e0cbb 2026-05-05 07:21:59,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:21:59,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f78c1f903fec0b0e0cbb HTTP/1.1" 200 None 2026-05-05 07:21:59,557 - lava_callback - DEBUG - Processing node 69f6f78c1f903fec0b0e0cbb: baseline-arm-clabbe 2026-05-05 07:21:59,557 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f78c1f903fec0b0e0cbb 2026-05-05 07:21:59,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:21:59,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f78c1f903fec0b0e0cbb HTTP/1.1" 200 None 2026-05-05 07:21:59,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:21:59,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:21:59,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:21:59,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f78c1f903fec0b0e0cbb HTTP/1.1" 200 None 2026-05-05 07:21:59,615 - lava_callback - ERROR - Error processing callback for node 69f6f78c1f903fec0b0e0cbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46602 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:22:39,505 - lava_callback - INFO - Received callback for node 69f6f7b11f903fec0b0e0ccb INFO: 172.20.0.1:46612 - "POST /node/69f6f7b11f903fec0b0e0ccb HTTP/1.0" 202 Accepted 2026-05-05 07:22:39,513 - lava_callback - INFO - Starting async job submit for node 69f6f7b11f903fec0b0e0ccb 2026-05-05 07:22:39,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:22:39,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7b11f903fec0b0e0ccb HTTP/1.1" 200 None 2026-05-05 07:22:39,528 - lava_callback - DEBUG - Processing node 69f6f7b11f903fec0b0e0ccb: baseline-arm-clabbe 2026-05-05 07:22:39,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f7b11f903fec0b0e0ccb 2026-05-05 07:22:39,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:22:39,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7b11f903fec0b0e0ccb HTTP/1.1" 200 None 2026-05-05 07:22:39,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:22:39,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:22:39,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:22:39,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f7b11f903fec0b0e0ccb HTTP/1.1" 200 None 2026-05-05 07:22:39,593 - lava_callback - ERROR - Error processing callback for node 69f6f7b11f903fec0b0e0ccb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58888 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:23:19,589 - lava_callback - INFO - Received callback for node 69f6f7d61f903fec0b0e0cdb INFO: 172.20.0.1:58904 - "POST /node/69f6f7d61f903fec0b0e0cdb HTTP/1.0" 202 Accepted 2026-05-05 07:23:19,595 - lava_callback - INFO - Starting async job submit for node 69f6f7d61f903fec0b0e0cdb 2026-05-05 07:23:19,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:19,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7d61f903fec0b0e0cdb HTTP/1.1" 200 None 2026-05-05 07:23:19,615 - lava_callback - DEBUG - Processing node 69f6f7d61f903fec0b0e0cdb: baseline-arm-clabbe 2026-05-05 07:23:19,615 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f7d61f903fec0b0e0cdb 2026-05-05 07:23:19,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:19,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7d61f903fec0b0e0cdb HTTP/1.1" 200 None 2026-05-05 07:23:19,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:19,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:23:19,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:19,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f7d61f903fec0b0e0cdb HTTP/1.1" 200 None 2026-05-05 07:23:19,679 - lava_callback - ERROR - Error processing callback for node 69f6f7d61f903fec0b0e0cdb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:23:47,317 - lava_callback - INFO - Received callback for node 69f6f7f81f903fec0b0e0ced INFO: 172.20.0.1:39036 - "POST /node/69f6f7f81f903fec0b0e0ced HTTP/1.0" 202 Accepted 2026-05-05 07:23:47,323 - lava_callback - INFO - Starting async job submit for node 69f6f7f81f903fec0b0e0ced 2026-05-05 07:23:47,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:47,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7f81f903fec0b0e0ced HTTP/1.1" 200 None 2026-05-05 07:23:47,342 - lava_callback - DEBUG - Processing node 69f6f7f81f903fec0b0e0ced: baseline-arm-baylibre 2026-05-05 07:23:47,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f7f81f903fec0b0e0ced 2026-05-05 07:23:47,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:47,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f7f81f903fec0b0e0ced HTTP/1.1" 200 None 2026-05-05 07:23:47,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:47,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:23:47,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:23:47,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f7f81f903fec0b0e0ced HTTP/1.1" 200 None 2026-05-05 07:23:47,404 - lava_callback - ERROR - Error processing callback for node 69f6f7f81f903fec0b0e0ced: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50322 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:25:07,488 - lava_callback - INFO - Received callback for node 69f6f8401f903fec0b0e0d11 INFO: 172.20.0.1:50324 - "POST /node/69f6f8401f903fec0b0e0d11 HTTP/1.0" 202 Accepted 2026-05-05 07:25:07,494 - lava_callback - INFO - Starting async job submit for node 69f6f8401f903fec0b0e0d11 2026-05-05 07:25:07,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:25:07,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f8401f903fec0b0e0d11 HTTP/1.1" 200 None 2026-05-05 07:25:07,509 - lava_callback - DEBUG - Processing node 69f6f8401f903fec0b0e0d11: baseline-arm-baylibre 2026-05-05 07:25:07,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f8401f903fec0b0e0d11 2026-05-05 07:25:07,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:25:07,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f8401f903fec0b0e0d11 HTTP/1.1" 200 None 2026-05-05 07:25:07,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:25:07,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:25:07,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:25:07,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f8401f903fec0b0e0d11 HTTP/1.1" 200 None 2026-05-05 07:25:07,563 - lava_callback - ERROR - Error processing callback for node 69f6f8401f903fec0b0e0d11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:26:19,555 - lava_callback - INFO - Received callback for node 69f6f88b1f903fec0b0e0d36 INFO: 172.20.0.1:38848 - "POST /node/69f6f88b1f903fec0b0e0d36 HTTP/1.0" 202 Accepted 2026-05-05 07:26:19,560 - lava_callback - INFO - Starting async job submit for node 69f6f88b1f903fec0b0e0d36 2026-05-05 07:26:19,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:26:19,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f88b1f903fec0b0e0d36 HTTP/1.1" 200 None 2026-05-05 07:26:19,588 - lava_callback - DEBUG - Processing node 69f6f88b1f903fec0b0e0d36: baseline-arm-clabbe 2026-05-05 07:26:19,588 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f88b1f903fec0b0e0d36 2026-05-05 07:26:19,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:26:19,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f88b1f903fec0b0e0d36 HTTP/1.1" 200 None 2026-05-05 07:26:19,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:26:19,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:26:19,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:26:19,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f88b1f903fec0b0e0d36 HTTP/1.1" 200 None 2026-05-05 07:26:19,632 - lava_callback - ERROR - Error processing callback for node 69f6f88b1f903fec0b0e0d36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60878 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:27:39,731 - lava_callback - INFO - Received callback for node 69f6f8d51f903fec0b0e0d5d INFO: 172.20.0.1:37632 - "POST /node/69f6f8d51f903fec0b0e0d5d HTTP/1.0" 202 Accepted 2026-05-05 07:27:39,738 - lava_callback - INFO - Starting async job submit for node 69f6f8d51f903fec0b0e0d5d 2026-05-05 07:27:39,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:27:39,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f8d51f903fec0b0e0d5d HTTP/1.1" 200 None 2026-05-05 07:27:39,754 - lava_callback - DEBUG - Processing node 69f6f8d51f903fec0b0e0d5d: baseline-arm-clabbe 2026-05-05 07:27:39,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f8d51f903fec0b0e0d5d 2026-05-05 07:27:39,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:27:39,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f8d51f903fec0b0e0d5d HTTP/1.1" 200 None 2026-05-05 07:27:39,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:27:39,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:27:39,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:27:39,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f8d51f903fec0b0e0d5d HTTP/1.1" 200 None 2026-05-05 07:27:39,807 - lava_callback - ERROR - Error processing callback for node 69f6f8d51f903fec0b0e0d5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41912 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:28:39,726 - lava_callback - INFO - Received callback for node 69f6f91f1f903fec0b0e0d80 INFO: 172.20.0.1:43220 - "POST /node/69f6f91f1f903fec0b0e0d80 HTTP/1.0" 202 Accepted 2026-05-05 07:28:39,732 - lava_callback - INFO - Starting async job submit for node 69f6f91f1f903fec0b0e0d80 2026-05-05 07:28:39,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:28:39,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f91f1f903fec0b0e0d80 HTTP/1.1" 200 None 2026-05-05 07:28:39,755 - lava_callback - DEBUG - Processing node 69f6f91f1f903fec0b0e0d80: baseline-arm-clabbe 2026-05-05 07:28:39,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f91f1f903fec0b0e0d80 2026-05-05 07:28:39,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:28:39,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f91f1f903fec0b0e0d80 HTTP/1.1" 200 None 2026-05-05 07:28:39,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:28:39,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:28:39,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:28:39,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f91f1f903fec0b0e0d80 HTTP/1.1" 200 None 2026-05-05 07:28:39,837 - lava_callback - ERROR - Error processing callback for node 69f6f91f1f903fec0b0e0d80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:29:19,720 - lava_callback - INFO - Received callback for node 69f6f9431f903fec0b0e0d93 INFO: 172.20.0.1:53278 - "POST /node/69f6f9431f903fec0b0e0d93 HTTP/1.0" 202 Accepted 2026-05-05 07:29:19,726 - lava_callback - INFO - Starting async job submit for node 69f6f9431f903fec0b0e0d93 2026-05-05 07:29:19,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:29:19,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f9431f903fec0b0e0d93 HTTP/1.1" 200 None 2026-05-05 07:29:19,746 - lava_callback - DEBUG - Processing node 69f6f9431f903fec0b0e0d93: baseline-arm-clabbe 2026-05-05 07:29:19,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f9431f903fec0b0e0d93 2026-05-05 07:29:19,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:29:19,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f9431f903fec0b0e0d93 HTTP/1.1" 200 None 2026-05-05 07:29:19,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:29:19,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:29:19,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:29:19,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f9431f903fec0b0e0d93 HTTP/1.1" 200 None 2026-05-05 07:29:19,799 - lava_callback - ERROR - Error processing callback for node 69f6f9431f903fec0b0e0d93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35476 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:30:07,821 - lava_callback - INFO - Received callback for node 69f6f96c1f903fec0b0e0da5 INFO: 172.20.0.1:35478 - "POST /node/69f6f96c1f903fec0b0e0da5 HTTP/1.0" 202 Accepted 2026-05-05 07:30:07,829 - lava_callback - INFO - Starting async job submit for node 69f6f96c1f903fec0b0e0da5 2026-05-05 07:30:07,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:30:07,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f96c1f903fec0b0e0da5 HTTP/1.1" 200 None 2026-05-05 07:30:07,877 - lava_callback - DEBUG - Processing node 69f6f96c1f903fec0b0e0da5: baseline-arm64-baylibre 2026-05-05 07:30:07,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f96c1f903fec0b0e0da5 2026-05-05 07:30:07,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:30:07,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f96c1f903fec0b0e0da5 HTTP/1.1" 200 None 2026-05-05 07:30:07,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:30:07,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:30:07,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:30:07,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f96c1f903fec0b0e0da5 HTTP/1.1" 200 None 2026-05-05 07:30:07,930 - lava_callback - ERROR - Error processing callback for node 69f6f96c1f903fec0b0e0da5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50778 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:31:20,080 - lava_callback - INFO - Received callback for node 69f6f9b11f903fec0b0e0dc7 INFO: 172.20.0.1:50050 - "POST /node/69f6f9b11f903fec0b0e0dc7 HTTP/1.0" 202 Accepted 2026-05-05 07:31:20,086 - lava_callback - INFO - Starting async job submit for node 69f6f9b11f903fec0b0e0dc7 2026-05-05 07:31:20,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:20,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f9b11f903fec0b0e0dc7 HTTP/1.1" 200 None 2026-05-05 07:31:20,108 - lava_callback - DEBUG - Processing node 69f6f9b11f903fec0b0e0dc7: baseline-arm-clabbe 2026-05-05 07:31:20,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f9b11f903fec0b0e0dc7 2026-05-05 07:31:20,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:20,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f9b11f903fec0b0e0dc7 HTTP/1.1" 200 None 2026-05-05 07:31:20,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:20,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:31:20,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:20,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f9b11f903fec0b0e0dc7 HTTP/1.1" 200 None 2026-05-05 07:31:20,178 - lava_callback - ERROR - Error processing callback for node 69f6f9b11f903fec0b0e0dc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:31:47,664 - lava_callback - INFO - Received callback for node 69f6f9dc1f903fec0b0e0de4 INFO: 172.20.0.1:51922 - "POST /node/69f6f9dc1f903fec0b0e0de4 HTTP/1.0" 202 Accepted 2026-05-05 07:31:47,670 - lava_callback - INFO - Starting async job submit for node 69f6f9dc1f903fec0b0e0de4 2026-05-05 07:31:47,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:47,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f9dc1f903fec0b0e0de4 HTTP/1.1" 200 None 2026-05-05 07:31:47,686 - lava_callback - DEBUG - Processing node 69f6f9dc1f903fec0b0e0de4: baseline-arm64-baylibre 2026-05-05 07:31:47,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6f9dc1f903fec0b0e0de4 2026-05-05 07:31:47,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:47,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6f9dc1f903fec0b0e0de4 HTTP/1.1" 200 None 2026-05-05 07:31:47,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:47,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:31:47,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:31:47,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6f9dc1f903fec0b0e0de4 HTTP/1.1" 200 None 2026-05-05 07:31:47,734 - lava_callback - ERROR - Error processing callback for node 69f6f9dc1f903fec0b0e0de4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54328 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:34:19,998 - lava_callback - INFO - Received callback for node 69f6fa681f903fec0b0e0e2b INFO: 172.20.0.1:47404 - "POST /node/69f6fa681f903fec0b0e0e2b HTTP/1.0" 202 Accepted 2026-05-05 07:34:20,005 - lava_callback - INFO - Starting async job submit for node 69f6fa681f903fec0b0e0e2b 2026-05-05 07:34:20,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:20,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fa681f903fec0b0e0e2b HTTP/1.1" 200 None 2026-05-05 07:34:20,021 - lava_callback - DEBUG - Processing node 69f6fa681f903fec0b0e0e2b: baseline-arm-clabbe 2026-05-05 07:34:20,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fa681f903fec0b0e0e2b 2026-05-05 07:34:20,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:20,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fa681f903fec0b0e0e2b HTTP/1.1" 200 None 2026-05-05 07:34:20,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:20,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e2b1c6f6c1a9f7050c0 HTTP/1.1" 200 None 2026-05-05 07:34:20,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:20,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fa681f903fec0b0e0e2b HTTP/1.1" 200 None 2026-05-05 07:34:20,087 - lava_callback - ERROR - Error processing callback for node 69f6fa681f903fec0b0e0e2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59814 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:34:47,878 - lava_callback - INFO - Received callback for node 69f6fa901f903fec0b0e0e41 INFO: 172.20.0.1:59816 - "POST /node/69f6fa901f903fec0b0e0e41 HTTP/1.0" 202 Accepted 2026-05-05 07:34:47,883 - lava_callback - INFO - Starting async job submit for node 69f6fa901f903fec0b0e0e41 2026-05-05 07:34:47,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:47,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fa901f903fec0b0e0e41 HTTP/1.1" 200 None 2026-05-05 07:34:47,900 - lava_callback - DEBUG - Processing node 69f6fa901f903fec0b0e0e41: baseline-arm64-baylibre 2026-05-05 07:34:47,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fa901f903fec0b0e0e41 2026-05-05 07:34:47,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:47,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fa901f903fec0b0e0e41 HTTP/1.1" 200 None 2026-05-05 07:34:47,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:47,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:34:47,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:34:47,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fa901f903fec0b0e0e41 HTTP/1.1" 200 None 2026-05-05 07:34:47,956 - lava_callback - ERROR - Error processing callback for node 69f6fa901f903fec0b0e0e41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42222 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:36:08,255 - lava_callback - INFO - Received callback for node 69f6fada1f903fec0b0e0e62 INFO: 172.20.0.1:42224 - "POST /node/69f6fada1f903fec0b0e0e62 HTTP/1.0" 202 Accepted 2026-05-05 07:36:08,261 - lava_callback - INFO - Starting async job submit for node 69f6fada1f903fec0b0e0e62 2026-05-05 07:36:08,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:36:08,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fada1f903fec0b0e0e62 HTTP/1.1" 200 None 2026-05-05 07:36:08,277 - lava_callback - DEBUG - Processing node 69f6fada1f903fec0b0e0e62: baseline-arm64-baylibre 2026-05-05 07:36:08,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fada1f903fec0b0e0e62 2026-05-05 07:36:08,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:36:08,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fada1f903fec0b0e0e62 HTTP/1.1" 200 None 2026-05-05 07:36:08,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:36:08,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:36:08,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:36:08,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fada1f903fec0b0e0e62 HTTP/1.1" 200 None 2026-05-05 07:36:08,343 - lava_callback - ERROR - Error processing callback for node 69f6fada1f903fec0b0e0e62: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39946 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:37:28,065 - lava_callback - INFO - Received callback for node 69f6fb231f903fec0b0e0e86 INFO: 172.20.0.1:39948 - "POST /node/69f6fb231f903fec0b0e0e86 HTTP/1.0" 202 Accepted 2026-05-05 07:37:28,071 - lava_callback - INFO - Starting async job submit for node 69f6fb231f903fec0b0e0e86 2026-05-05 07:37:28,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:37:28,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fb231f903fec0b0e0e86 HTTP/1.1" 200 None 2026-05-05 07:37:28,087 - lava_callback - DEBUG - Processing node 69f6fb231f903fec0b0e0e86: baseline-arm64-baylibre 2026-05-05 07:37:28,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fb231f903fec0b0e0e86 2026-05-05 07:37:28,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:37:28,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fb231f903fec0b0e0e86 HTTP/1.1" 200 None 2026-05-05 07:37:28,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:37:28,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:37:28,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:37:28,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fb231f903fec0b0e0e86 HTTP/1.1" 200 None 2026-05-05 07:37:28,134 - lava_callback - ERROR - Error processing callback for node 69f6fb231f903fec0b0e0e86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35158 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:38:28,076 - lava_callback - INFO - Received callback for node 69f6fb6c1f903fec0b0e0eb5 INFO: 172.20.0.1:35164 - "POST /node/69f6fb6c1f903fec0b0e0eb5 HTTP/1.0" 202 Accepted 2026-05-05 07:38:28,082 - lava_callback - INFO - Starting async job submit for node 69f6fb6c1f903fec0b0e0eb5 2026-05-05 07:38:28,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:38:28,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fb6c1f903fec0b0e0eb5 HTTP/1.1" 200 None 2026-05-05 07:38:28,102 - lava_callback - DEBUG - Processing node 69f6fb6c1f903fec0b0e0eb5: baseline-arm64-baylibre 2026-05-05 07:38:28,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fb6c1f903fec0b0e0eb5 2026-05-05 07:38:28,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:38:28,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fb6c1f903fec0b0e0eb5 HTTP/1.1" 200 None 2026-05-05 07:38:28,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:38:28,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:38:28,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:38:28,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fb6c1f903fec0b0e0eb5 HTTP/1.1" 200 None 2026-05-05 07:38:28,156 - lava_callback - ERROR - Error processing callback for node 69f6fb6c1f903fec0b0e0eb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48328 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:40:00,173 - lava_callback - INFO - Received callback for node 69f6fbb91f903fec0b0e0ed5 INFO: 172.20.0.1:33966 - "POST /node/69f6fbb91f903fec0b0e0ed5 HTTP/1.0" 202 Accepted 2026-05-05 07:40:00,179 - lava_callback - INFO - Starting async job submit for node 69f6fbb91f903fec0b0e0ed5 2026-05-05 07:40:00,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:00,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fbb91f903fec0b0e0ed5 HTTP/1.1" 200 None 2026-05-05 07:40:00,195 - lava_callback - DEBUG - Processing node 69f6fbb91f903fec0b0e0ed5: baseline-arm64-clabbe 2026-05-05 07:40:00,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fbb91f903fec0b0e0ed5 2026-05-05 07:40:00,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:00,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fbb91f903fec0b0e0ed5 HTTP/1.1" 200 None 2026-05-05 07:40:00,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:00,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:40:00,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:00,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fbb91f903fec0b0e0ed5 HTTP/1.1" 200 None 2026-05-05 07:40:00,252 - lava_callback - ERROR - Error processing callback for node 69f6fbb91f903fec0b0e0ed5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:40:48,275 - lava_callback - INFO - Received callback for node 69f6fbfc1f903fec0b0e0f04 INFO: 172.20.0.1:37588 - "POST /node/69f6fbfc1f903fec0b0e0f04 HTTP/1.0" 202 Accepted 2026-05-05 07:40:48,281 - lava_callback - INFO - Starting async job submit for node 69f6fbfc1f903fec0b0e0f04 2026-05-05 07:40:48,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:48,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fbfc1f903fec0b0e0f04 HTTP/1.1" 200 None 2026-05-05 07:40:48,296 - lava_callback - DEBUG - Processing node 69f6fbfc1f903fec0b0e0f04: baseline-arm64-baylibre 2026-05-05 07:40:48,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fbfc1f903fec0b0e0f04 2026-05-05 07:40:48,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:48,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fbfc1f903fec0b0e0f04 HTTP/1.1" 200 None 2026-05-05 07:40:48,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:48,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:40:48,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:40:48,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fbfc1f903fec0b0e0f04 HTTP/1.1" 200 None 2026-05-05 07:40:48,355 - lava_callback - ERROR - Error processing callback for node 69f6fbfc1f903fec0b0e0f04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47694 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:42:20,289 - lava_callback - INFO - Received callback for node 69f6fc4a1f903fec0b0e0f26 INFO: 172.20.0.1:39582 - "POST /node/69f6fc4a1f903fec0b0e0f26 HTTP/1.0" 202 Accepted 2026-05-05 07:42:20,294 - lava_callback - INFO - Starting async job submit for node 69f6fc4a1f903fec0b0e0f26 2026-05-05 07:42:20,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:42:20,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fc4a1f903fec0b0e0f26 HTTP/1.1" 200 None 2026-05-05 07:42:20,311 - lava_callback - DEBUG - Processing node 69f6fc4a1f903fec0b0e0f26: baseline-arm64-clabbe 2026-05-05 07:42:20,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fc4a1f903fec0b0e0f26 2026-05-05 07:42:20,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:42:20,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fc4a1f903fec0b0e0f26 HTTP/1.1" 200 None 2026-05-05 07:42:20,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:42:20,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:42:20,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:42:20,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fc4a1f903fec0b0e0f26 HTTP/1.1" 200 None 2026-05-05 07:42:20,358 - lava_callback - ERROR - Error processing callback for node 69f6fc4a1f903fec0b0e0f26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53104 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:43:20,277 - lava_callback - INFO - Received callback for node 69f6fc941f903fec0b0e0f57 INFO: 172.20.0.1:46204 - "POST /node/69f6fc941f903fec0b0e0f57 HTTP/1.0" 202 Accepted 2026-05-05 07:43:20,282 - lava_callback - INFO - Starting async job submit for node 69f6fc941f903fec0b0e0f57 2026-05-05 07:43:20,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:43:20,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fc941f903fec0b0e0f57 HTTP/1.1" 200 None 2026-05-05 07:43:20,297 - lava_callback - DEBUG - Processing node 69f6fc941f903fec0b0e0f57: baseline-arm64-clabbe 2026-05-05 07:43:20,298 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fc941f903fec0b0e0f57 2026-05-05 07:43:20,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:43:20,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fc941f903fec0b0e0f57 HTTP/1.1" 200 None 2026-05-05 07:43:20,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:43:20,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:43:20,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:43:20,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fc941f903fec0b0e0f57 HTTP/1.1" 200 None 2026-05-05 07:43:20,343 - lava_callback - ERROR - Error processing callback for node 69f6fc941f903fec0b0e0f57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:44:40,347 - lava_callback - INFO - Received callback for node 69f6fcdd1f903fec0b0e0f76 INFO: 172.20.0.1:49840 - "POST /node/69f6fcdd1f903fec0b0e0f76 HTTP/1.0" 202 Accepted 2026-05-05 07:44:40,356 - lava_callback - INFO - Starting async job submit for node 69f6fcdd1f903fec0b0e0f76 2026-05-05 07:44:40,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:44:40,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fcdd1f903fec0b0e0f76 HTTP/1.1" 200 None 2026-05-05 07:44:40,376 - lava_callback - DEBUG - Processing node 69f6fcdd1f903fec0b0e0f76: baseline-arm64-clabbe 2026-05-05 07:44:40,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fcdd1f903fec0b0e0f76 2026-05-05 07:44:40,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:44:40,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fcdd1f903fec0b0e0f76 HTTP/1.1" 200 None 2026-05-05 07:44:40,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:44:40,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:44:40,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:44:40,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fcdd1f903fec0b0e0f76 HTTP/1.1" 200 None 2026-05-05 07:44:40,452 - lava_callback - ERROR - Error processing callback for node 69f6fcdd1f903fec0b0e0f76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39136 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:46:00,439 - lava_callback - INFO - Received callback for node 69f6fd261f903fec0b0e0fa9 INFO: 172.20.0.1:45344 - "POST /node/69f6fd261f903fec0b0e0fa9 HTTP/1.0" 202 Accepted 2026-05-05 07:46:00,448 - lava_callback - INFO - Starting async job submit for node 69f6fd261f903fec0b0e0fa9 2026-05-05 07:46:00,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:46:00,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fd261f903fec0b0e0fa9 HTTP/1.1" 200 None 2026-05-05 07:46:00,466 - lava_callback - DEBUG - Processing node 69f6fd261f903fec0b0e0fa9: baseline-arm64-clabbe 2026-05-05 07:46:00,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fd261f903fec0b0e0fa9 2026-05-05 07:46:00,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:46:00,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fd261f903fec0b0e0fa9 HTTP/1.1" 200 None 2026-05-05 07:46:00,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:46:00,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:46:00,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:46:00,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fd261f903fec0b0e0fa9 HTTP/1.1" 200 None 2026-05-05 07:46:00,522 - lava_callback - ERROR - Error processing callback for node 69f6fd261f903fec0b0e0fa9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:47:00,437 - lava_callback - INFO - Received callback for node 69f6fd6f1f903fec0b0e0fc7 INFO: 172.20.0.1:43562 - "POST /node/69f6fd6f1f903fec0b0e0fc7 HTTP/1.0" 202 Accepted 2026-05-05 07:47:00,443 - lava_callback - INFO - Starting async job submit for node 69f6fd6f1f903fec0b0e0fc7 2026-05-05 07:47:00,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:47:00,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fd6f1f903fec0b0e0fc7 HTTP/1.1" 200 None 2026-05-05 07:47:00,469 - lava_callback - DEBUG - Processing node 69f6fd6f1f903fec0b0e0fc7: baseline-arm64-clabbe 2026-05-05 07:47:00,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fd6f1f903fec0b0e0fc7 2026-05-05 07:47:00,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:47:00,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fd6f1f903fec0b0e0fc7 HTTP/1.1" 200 None 2026-05-05 07:47:00,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:47:00,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:47:00,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:47:00,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fd6f1f903fec0b0e0fc7 HTTP/1.1" 200 None 2026-05-05 07:47:00,517 - lava_callback - ERROR - Error processing callback for node 69f6fd6f1f903fec0b0e0fc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:48:20,494 - lava_callback - INFO - Received callback for node 69f6fdb81f903fec0b0e0fe9 INFO: 172.20.0.1:45954 - "POST /node/69f6fdb81f903fec0b0e0fe9 HTTP/1.0" 202 Accepted 2026-05-05 07:48:20,502 - lava_callback - INFO - Starting async job submit for node 69f6fdb81f903fec0b0e0fe9 2026-05-05 07:48:20,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:48:20,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fdb81f903fec0b0e0fe9 HTTP/1.1" 200 None 2026-05-05 07:48:20,518 - lava_callback - DEBUG - Processing node 69f6fdb81f903fec0b0e0fe9: baseline-arm64-clabbe 2026-05-05 07:48:20,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fdb81f903fec0b0e0fe9 2026-05-05 07:48:20,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:48:20,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fdb81f903fec0b0e0fe9 HTTP/1.1" 200 None 2026-05-05 07:48:20,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:48:20,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:48:20,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:48:20,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fdb81f903fec0b0e0fe9 HTTP/1.1" 200 None 2026-05-05 07:48:20,575 - lava_callback - ERROR - Error processing callback for node 69f6fdb81f903fec0b0e0fe9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34094 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:50:40,649 - lava_callback - INFO - Received callback for node 69f6fe491f903fec0b0e103b INFO: 172.20.0.1:36214 - "POST /node/69f6fe491f903fec0b0e103b HTTP/1.0" 202 Accepted 2026-05-05 07:50:40,655 - lava_callback - INFO - Starting async job submit for node 69f6fe491f903fec0b0e103b 2026-05-05 07:50:40,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:50:40,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fe491f903fec0b0e103b HTTP/1.1" 200 None 2026-05-05 07:50:40,671 - lava_callback - DEBUG - Processing node 69f6fe491f903fec0b0e103b: baseline-arm64-clabbe 2026-05-05 07:50:40,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fe491f903fec0b0e103b 2026-05-05 07:50:40,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:50:40,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fe491f903fec0b0e103b HTTP/1.1" 200 None 2026-05-05 07:50:40,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:50:40,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:50:40,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:50:40,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fe491f903fec0b0e103b HTTP/1.1" 200 None 2026-05-05 07:50:40,733 - lava_callback - ERROR - Error processing callback for node 69f6fe491f903fec0b0e103b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52164 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:53:00,651 - lava_callback - INFO - Received callback for node 69f6feda1f903fec0b0e108c INFO: 172.20.0.1:48908 - "POST /node/69f6feda1f903fec0b0e108c HTTP/1.0" 202 Accepted 2026-05-05 07:53:00,660 - lava_callback - INFO - Starting async job submit for node 69f6feda1f903fec0b0e108c 2026-05-05 07:53:00,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:53:00,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6feda1f903fec0b0e108c HTTP/1.1" 200 None 2026-05-05 07:53:00,683 - lava_callback - DEBUG - Processing node 69f6feda1f903fec0b0e108c: baseline-arm64-clabbe 2026-05-05 07:53:00,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6feda1f903fec0b0e108c 2026-05-05 07:53:00,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:53:00,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6feda1f903fec0b0e108c HTTP/1.1" 200 None 2026-05-05 07:53:00,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:53:00,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:53:00,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:53:00,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6feda1f903fec0b0e108c HTTP/1.1" 200 None 2026-05-05 07:53:00,737 - lava_callback - ERROR - Error processing callback for node 69f6feda1f903fec0b0e108c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41632 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:54:20,714 - lava_callback - INFO - Received callback for node 69f6ff221f903fec0b0e10ba INFO: 172.20.0.1:33674 - "POST /node/69f6ff221f903fec0b0e10ba HTTP/1.0" 202 Accepted 2026-05-05 07:54:20,720 - lava_callback - INFO - Starting async job submit for node 69f6ff221f903fec0b0e10ba 2026-05-05 07:54:20,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:54:20,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ff221f903fec0b0e10ba HTTP/1.1" 200 None 2026-05-05 07:54:20,745 - lava_callback - DEBUG - Processing node 69f6ff221f903fec0b0e10ba: baseline-arm64-clabbe 2026-05-05 07:54:20,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ff221f903fec0b0e10ba 2026-05-05 07:54:20,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:54:20,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ff221f903fec0b0e10ba HTTP/1.1" 200 None 2026-05-05 07:54:20,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:54:20,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:54:20,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:54:20,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ff221f903fec0b0e10ba HTTP/1.1" 200 None 2026-05-05 07:54:20,805 - lava_callback - ERROR - Error processing callback for node 69f6ff221f903fec0b0e10ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:55:40,802 - lava_callback - INFO - Received callback for node 69f6ff6c1f903fec0b0e10da INFO: 172.20.0.1:60294 - "POST /node/69f6ff6c1f903fec0b0e10da HTTP/1.0" 202 Accepted 2026-05-05 07:55:40,807 - lava_callback - INFO - Starting async job submit for node 69f6ff6c1f903fec0b0e10da 2026-05-05 07:55:40,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:55:40,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ff6c1f903fec0b0e10da HTTP/1.1" 200 None 2026-05-05 07:55:40,846 - lava_callback - DEBUG - Processing node 69f6ff6c1f903fec0b0e10da: baseline-arm64-clabbe 2026-05-05 07:55:40,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ff6c1f903fec0b0e10da 2026-05-05 07:55:40,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:55:40,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ff6c1f903fec0b0e10da HTTP/1.1" 200 None 2026-05-05 07:55:40,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:55:40,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:55:40,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:55:40,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ff6c1f903fec0b0e10da HTTP/1.1" 200 None 2026-05-05 07:55:40,923 - lava_callback - ERROR - Error processing callback for node 69f6ff6c1f903fec0b0e10da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36238 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:56:40,821 - lava_callback - INFO - Received callback for node 69f6ffb51f903fec0b0e10f9 INFO: 172.20.0.1:58638 - "POST /node/69f6ffb51f903fec0b0e10f9 HTTP/1.0" 202 Accepted 2026-05-05 07:56:40,827 - lava_callback - INFO - Starting async job submit for node 69f6ffb51f903fec0b0e10f9 2026-05-05 07:56:40,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:56:40,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ffb51f903fec0b0e10f9 HTTP/1.1" 200 None 2026-05-05 07:56:40,843 - lava_callback - DEBUG - Processing node 69f6ffb51f903fec0b0e10f9: baseline-arm64-clabbe 2026-05-05 07:56:40,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6ffb51f903fec0b0e10f9 2026-05-05 07:56:40,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:56:40,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6ffb51f903fec0b0e10f9 HTTP/1.1" 200 None 2026-05-05 07:56:40,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:56:40,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:56:40,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:56:40,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6ffb51f903fec0b0e10f9 HTTP/1.1" 200 None 2026-05-05 07:56:40,900 - lava_callback - ERROR - Error processing callback for node 69f6ffb51f903fec0b0e10f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49208 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:57:49,290 - lava_callback - INFO - Received callback for node 69f6fff71f903fec0b0e1125 INFO: 172.20.0.1:49212 - "POST /node/69f6fff71f903fec0b0e1125 HTTP/1.0" 202 Accepted 2026-05-05 07:57:49,296 - lava_callback - INFO - Starting async job submit for node 69f6fff71f903fec0b0e1125 2026-05-05 07:57:49,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:57:49,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fff71f903fec0b0e1125 HTTP/1.1" 200 None 2026-05-05 07:57:49,310 - lava_callback - DEBUG - Processing node 69f6fff71f903fec0b0e1125: baseline-arm64-baylibre 2026-05-05 07:57:49,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f6fff71f903fec0b0e1125 2026-05-05 07:57:49,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:57:49,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f6fff71f903fec0b0e1125 HTTP/1.1" 200 None 2026-05-05 07:57:49,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:57:49,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:57:49,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:57:49,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f6fff71f903fec0b0e1125 HTTP/1.1" 200 None 2026-05-05 07:57:49,356 - lava_callback - ERROR - Error processing callback for node 69f6fff71f903fec0b0e1125: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41934 - "GET / HTTP/1.0" 200 OK 2026-05-05 07:59:09,174 - lava_callback - INFO - Received callback for node 69f7003f1f903fec0b0e1146 INFO: 172.20.0.1:41936 - "POST /node/69f7003f1f903fec0b0e1146 HTTP/1.0" 202 Accepted 2026-05-05 07:59:09,179 - lava_callback - INFO - Starting async job submit for node 69f7003f1f903fec0b0e1146 2026-05-05 07:59:09,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:59:09,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f7003f1f903fec0b0e1146 HTTP/1.1" 200 None 2026-05-05 07:59:09,195 - lava_callback - DEBUG - Processing node 69f7003f1f903fec0b0e1146: baseline-arm64-baylibre 2026-05-05 07:59:09,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f7003f1f903fec0b0e1146 2026-05-05 07:59:09,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:59:09,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f7003f1f903fec0b0e1146 HTTP/1.1" 200 None 2026-05-05 07:59:09,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:59:09,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 07:59:09,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 07:59:09,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f7003f1f903fec0b0e1146 HTTP/1.1" 200 None 2026-05-05 07:59:09,252 - lava_callback - ERROR - Error processing callback for node 69f7003f1f903fec0b0e1146: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34838 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:00:29,234 - lava_callback - INFO - Received callback for node 69f700881f903fec0b0e1168 INFO: 172.20.0.1:34844 - "POST /node/69f700881f903fec0b0e1168 HTTP/1.0" 202 Accepted 2026-05-05 08:00:29,240 - lava_callback - INFO - Starting async job submit for node 69f700881f903fec0b0e1168 2026-05-05 08:00:29,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:00:29,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f700881f903fec0b0e1168 HTTP/1.1" 200 None 2026-05-05 08:00:29,267 - lava_callback - DEBUG - Processing node 69f700881f903fec0b0e1168: baseline-arm64-baylibre 2026-05-05 08:00:29,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f700881f903fec0b0e1168 2026-05-05 08:00:29,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:00:29,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f700881f903fec0b0e1168 HTTP/1.1" 200 None 2026-05-05 08:00:29,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:00:29,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:00:29,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:00:29,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f700881f903fec0b0e1168 HTTP/1.1" 200 None 2026-05-05 08:00:29,328 - lava_callback - ERROR - Error processing callback for node 69f700881f903fec0b0e1168: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46302 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:01:29,272 - lava_callback - INFO - Received callback for node 69f700d41f903fec0b0e1197 INFO: 172.20.0.1:46306 - "POST /node/69f700d41f903fec0b0e1197 HTTP/1.0" 202 Accepted 2026-05-05 08:01:29,280 - lava_callback - INFO - Starting async job submit for node 69f700d41f903fec0b0e1197 2026-05-05 08:01:29,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:01:29,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f700d41f903fec0b0e1197 HTTP/1.1" 200 None 2026-05-05 08:01:29,299 - lava_callback - DEBUG - Processing node 69f700d41f903fec0b0e1197: baseline-arm64-baylibre 2026-05-05 08:01:29,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f700d41f903fec0b0e1197 2026-05-05 08:01:29,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:01:29,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f700d41f903fec0b0e1197 HTTP/1.1" 200 None 2026-05-05 08:01:29,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:01:29,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:01:29,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:01:29,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f700d41f903fec0b0e1197 HTTP/1.1" 200 None 2026-05-05 08:01:29,361 - lava_callback - ERROR - Error processing callback for node 69f700d41f903fec0b0e1197: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39204 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:02:49,425 - lava_callback - INFO - Received callback for node 69f7011c1f903fec0b0e11bd INFO: 172.20.0.1:39220 - "POST /node/69f7011c1f903fec0b0e11bd HTTP/1.0" 202 Accepted 2026-05-05 08:02:49,431 - lava_callback - INFO - Starting async job submit for node 69f7011c1f903fec0b0e11bd 2026-05-05 08:02:49,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:02:49,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f7011c1f903fec0b0e11bd HTTP/1.1" 200 None 2026-05-05 08:02:49,446 - lava_callback - DEBUG - Processing node 69f7011c1f903fec0b0e11bd: baseline-arm64-baylibre 2026-05-05 08:02:49,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f7011c1f903fec0b0e11bd 2026-05-05 08:02:49,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:02:49,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f7011c1f903fec0b0e11bd HTTP/1.1" 200 None 2026-05-05 08:02:49,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:02:49,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:02:49,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:02:49,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f7011c1f903fec0b0e11bd HTTP/1.1" 200 None 2026-05-05 08:02:49,492 - lava_callback - ERROR - Error processing callback for node 69f7011c1f903fec0b0e11bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52062 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:04:09,543 - lava_callback - INFO - Received callback for node 69f701651f903fec0b0e11ee INFO: 172.20.0.1:52064 - "POST /node/69f701651f903fec0b0e11ee HTTP/1.0" 202 Accepted 2026-05-05 08:04:09,551 - lava_callback - INFO - Starting async job submit for node 69f701651f903fec0b0e11ee 2026-05-05 08:04:09,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:04:09,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f701651f903fec0b0e11ee HTTP/1.1" 200 None 2026-05-05 08:04:09,569 - lava_callback - DEBUG - Processing node 69f701651f903fec0b0e11ee: baseline-arm64-baylibre 2026-05-05 08:04:09,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f701651f903fec0b0e11ee 2026-05-05 08:04:09,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:04:09,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f701651f903fec0b0e11ee HTTP/1.1" 200 None 2026-05-05 08:04:09,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:04:09,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:04:09,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:04:09,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f701651f903fec0b0e11ee HTTP/1.1" 200 None 2026-05-05 08:04:09,624 - lava_callback - ERROR - Error processing callback for node 69f701651f903fec0b0e11ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56378 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:05:21,456 - lava_callback - INFO - Received callback for node 69f701b31f903fec0b0e1214 INFO: 172.20.0.1:47486 - "POST /node/69f701b31f903fec0b0e1214 HTTP/1.0" 202 Accepted 2026-05-05 08:05:21,461 - lava_callback - INFO - Starting async job submit for node 69f701b31f903fec0b0e1214 2026-05-05 08:05:21,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:05:21,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f701b31f903fec0b0e1214 HTTP/1.1" 200 None 2026-05-05 08:05:21,482 - lava_callback - DEBUG - Processing node 69f701b31f903fec0b0e1214: baseline-arm64-clabbe 2026-05-05 08:05:21,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f701b31f903fec0b0e1214 2026-05-05 08:05:21,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:05:21,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f701b31f903fec0b0e1214 HTTP/1.1" 200 None 2026-05-05 08:05:21,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:05:21,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:05:21,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:05:21,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f701b31f903fec0b0e1214 HTTP/1.1" 200 None 2026-05-05 08:05:21,528 - lava_callback - ERROR - Error processing callback for node 69f701b31f903fec0b0e1214: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:06:41,241 - lava_callback - INFO - Received callback for node 69f701fd1f903fec0b0e1246 INFO: 172.20.0.1:39594 - "POST /node/69f701fd1f903fec0b0e1246 HTTP/1.0" 202 Accepted 2026-05-05 08:06:41,247 - lava_callback - INFO - Starting async job submit for node 69f701fd1f903fec0b0e1246 2026-05-05 08:06:41,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:06:41,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f701fd1f903fec0b0e1246 HTTP/1.1" 200 None 2026-05-05 08:06:41,268 - lava_callback - DEBUG - Processing node 69f701fd1f903fec0b0e1246: baseline-arm64-clabbe 2026-05-05 08:06:41,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f701fd1f903fec0b0e1246 2026-05-05 08:06:41,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:06:41,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f701fd1f903fec0b0e1246 HTTP/1.1" 200 None 2026-05-05 08:06:41,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:06:41,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:06:41,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:06:41,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f701fd1f903fec0b0e1246 HTTP/1.1" 200 None 2026-05-05 08:06:41,338 - lava_callback - ERROR - Error processing callback for node 69f701fd1f903fec0b0e1246: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:07:41,329 - lava_callback - INFO - Received callback for node 69f702471f903fec0b0e126c INFO: 172.20.0.1:34526 - "POST /node/69f702471f903fec0b0e126c HTTP/1.0" 202 Accepted 2026-05-05 08:07:41,335 - lava_callback - INFO - Starting async job submit for node 69f702471f903fec0b0e126c 2026-05-05 08:07:41,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:07:41,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f702471f903fec0b0e126c HTTP/1.1" 200 None 2026-05-05 08:07:41,352 - lava_callback - DEBUG - Processing node 69f702471f903fec0b0e126c: baseline-arm64-clabbe 2026-05-05 08:07:41,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f702471f903fec0b0e126c 2026-05-05 08:07:41,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:07:41,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f702471f903fec0b0e126c HTTP/1.1" 200 None 2026-05-05 08:07:41,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:07:41,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:07:41,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:07:41,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f702471f903fec0b0e126c HTTP/1.1" 200 None 2026-05-05 08:07:41,412 - lava_callback - ERROR - Error processing callback for node 69f702471f903fec0b0e126c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:09:01,324 - lava_callback - INFO - Received callback for node 69f702911f903fec0b0e12a1 INFO: 172.20.0.1:55130 - "POST /node/69f702911f903fec0b0e12a1 HTTP/1.0" 202 Accepted 2026-05-05 08:09:01,329 - lava_callback - INFO - Starting async job submit for node 69f702911f903fec0b0e12a1 2026-05-05 08:09:01,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:09:01,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f702911f903fec0b0e12a1 HTTP/1.1" 200 None 2026-05-05 08:09:01,345 - lava_callback - DEBUG - Processing node 69f702911f903fec0b0e12a1: baseline-arm64-clabbe 2026-05-05 08:09:01,345 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f702911f903fec0b0e12a1 2026-05-05 08:09:01,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:09:01,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f702911f903fec0b0e12a1 HTTP/1.1" 200 None 2026-05-05 08:09:01,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:09:01,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:09:01,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:09:01,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f702911f903fec0b0e12a1 HTTP/1.1" 200 None 2026-05-05 08:09:01,402 - lava_callback - ERROR - Error processing callback for node 69f702911f903fec0b0e12a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38464 - "GET / HTTP/1.0" 200 OK 2026-05-05 08:12:41,457 - lava_callback - INFO - Received callback for node 69f703711f903fec0b0e12ef INFO: 172.20.0.1:53522 - "POST /node/69f703711f903fec0b0e12ef HTTP/1.0" 202 Accepted 2026-05-05 08:12:41,463 - lava_callback - INFO - Starting async job submit for node 69f703711f903fec0b0e12ef 2026-05-05 08:12:41,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:12:41,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f703711f903fec0b0e12ef HTTP/1.1" 200 None 2026-05-05 08:12:41,478 - lava_callback - DEBUG - Processing node 69f703711f903fec0b0e12ef: baseline-arm64-clabbe 2026-05-05 08:12:41,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69f703711f903fec0b0e12ef 2026-05-05 08:12:41,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:12:41,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f703711f903fec0b0e12ef HTTP/1.1" 200 None 2026-05-05 08:12:41,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:12:41,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69f14e391c6f6c1a9f7050cf HTTP/1.1" 200 None 2026-05-05 08:12:41,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-05-05 08:12:41,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69f703711f903fec0b0e12ef HTTP/1.1" 200 None 2026-05-05 08:12:41,523 - lava_callback - ERROR - Error processing callback for node 69f703711f903fec0b0e12ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 373, in async_job_submit _emit_callback_telemetry(job_node, job_callback, hierarchy) File "/home/kernelci/pipeline/lava_callback.py", line 393, in _emit_callback_telemetry job_callback.is_infra_error() File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 117, in is_infra_error lava_yaml = self._data["results"]["lava"] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]