INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:32:57,995 - lava_callback - INFO - Received callback for node 69c47e48590270ad7b6d8ff2 2026-03-26 00:32:58,002 - lava_callback - INFO - Starting async job submit for node 69c47e48590270ad7b6d8ff2 INFO: 172.21.0.1:42960 - "POST /node/69c47e48590270ad7b6d8ff2 HTTP/1.0" 202 Accepted 2026-03-26 00:32:58,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:32:58,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e48590270ad7b6d8ff2 HTTP/1.1" 200 None 2026-03-26 00:32:58,045 - lava_callback - DEBUG - Processing node 69c47e48590270ad7b6d8ff2: baseline-arm 2026-03-26 00:32:58,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:32:58,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:32:58,397 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c47e48590270ad7b6d8ff2/log.txt.gz 2026-03-26 00:32:58,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:32:58,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:32:58,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c47e48590270ad7b6d8ff2/lava-logs.yaml 2026-03-26 00:32:58,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:32:58,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:32:58,903 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c47e48590270ad7b6d8ff2/lava_callback.json.gz 2026-03-26 00:32:58,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:32:58,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e48590270ad7b6d8ff2 HTTP/1.1" 200 None 2026-03-26 00:32:58,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:32:58,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b41590270ad7b6d6994 HTTP/1.1" 200 None 2026-03-26 00:32:58,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:32:58,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47e48590270ad7b6d8ff2 HTTP/1.1" 200 None 2026-03-26 00:32:58,992 - lava_callback - INFO - Completed processing callback for node 69c47e48590270ad7b6d8ff2 INFO: 172.21.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:33:28,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:33:29,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34870 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:36:49,369 - lava_callback - INFO - Received callback for node 69c1dc960efaf21d931035b7 INFO: 172.21.0.1:34882 - "POST /node/69c1dc960efaf21d931035b7 HTTP/1.0" 202 Accepted 2026-03-26 00:36:49,374 - lava_callback - INFO - Starting async job submit for node 69c1dc960efaf21d931035b7 2026-03-26 00:36:49,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:49,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc960efaf21d931035b7 HTTP/1.1" 200 None 2026-03-26 00:36:49,415 - lava_callback - DEBUG - Processing node 69c1dc960efaf21d931035b7: baseline-arm-baylibre 2026-03-26 00:36:49,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dc960efaf21d931035b7 2026-03-26 00:36:49,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:49,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc960efaf21d931035b7 HTTP/1.1" 200 None 2026-03-26 00:36:49,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:49,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:36:49,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:49,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc960efaf21d931035b7 HTTP/1.1" 200 None 2026-03-26 00:36:49,477 - lava_callback - ERROR - Error processing callback for node 69c1dc960efaf21d931035b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:36:50,057 - lava_callback - INFO - Received callback for node 69c1dc980efaf21d931035ba INFO: 172.21.0.1:33090 - "POST /node/69c1dc980efaf21d931035ba HTTP/1.0" 202 Accepted 2026-03-26 00:36:50,064 - lava_callback - INFO - Starting async job submit for node 69c1dc980efaf21d931035ba 2026-03-26 00:36:50,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc980efaf21d931035ba HTTP/1.1" 200 None 2026-03-26 00:36:50,080 - lava_callback - DEBUG - Processing node 69c1dc980efaf21d931035ba: baseline-arm-baylibre 2026-03-26 00:36:50,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dc980efaf21d931035ba 2026-03-26 00:36:50,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc980efaf21d931035ba HTTP/1.1" 200 None 2026-03-26 00:36:50,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:36:50,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc980efaf21d931035ba HTTP/1.1" 200 None 2026-03-26 00:36:50,144 - lava_callback - ERROR - Error processing callback for node 69c1dc980efaf21d931035ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:36:50,744 - lava_callback - INFO - Received callback for node 69c1dc990efaf21d931035bd INFO: 172.21.0.1:33098 - "POST /node/69c1dc990efaf21d931035bd HTTP/1.0" 202 Accepted 2026-03-26 00:36:50,750 - lava_callback - INFO - Starting async job submit for node 69c1dc990efaf21d931035bd 2026-03-26 00:36:50,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc990efaf21d931035bd HTTP/1.1" 200 None 2026-03-26 00:36:50,766 - lava_callback - DEBUG - Processing node 69c1dc990efaf21d931035bd: baseline-arm-baylibre 2026-03-26 00:36:50,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dc990efaf21d931035bd 2026-03-26 00:36:50,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc990efaf21d931035bd HTTP/1.1" 200 None 2026-03-26 00:36:50,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:36:50,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:50,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc990efaf21d931035bd HTTP/1.1" 200 None 2026-03-26 00:36:50,833 - lava_callback - ERROR - Error processing callback for node 69c1dc990efaf21d931035bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:36:51,407 - lava_callback - INFO - Received callback for node 69c1dc9a0efaf21d931035cd INFO: 172.21.0.1:33112 - "POST /node/69c1dc9a0efaf21d931035cd HTTP/1.0" 202 Accepted 2026-03-26 00:36:51,417 - lava_callback - INFO - Starting async job submit for node 69c1dc9a0efaf21d931035cd 2026-03-26 00:36:51,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:51,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9a0efaf21d931035cd HTTP/1.1" 200 None 2026-03-26 00:36:51,451 - lava_callback - DEBUG - Processing node 69c1dc9a0efaf21d931035cd: baseline-arm-baylibre 2026-03-26 00:36:51,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dc9a0efaf21d931035cd 2026-03-26 00:36:51,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:51,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9a0efaf21d931035cd HTTP/1.1" 200 None 2026-03-26 00:36:51,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:51,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:36:51,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:51,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc9a0efaf21d931035cd HTTP/1.1" 200 None 2026-03-26 00:36:51,542 - lava_callback - ERROR - Error processing callback for node 69c1dc9a0efaf21d931035cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33120 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:36:52,080 - lava_callback - INFO - Received callback for node 69c1dc9d0efaf21d931035dd INFO: 172.21.0.1:33136 - "POST /node/69c1dc9d0efaf21d931035dd HTTP/1.0" 202 Accepted 2026-03-26 00:36:52,086 - lava_callback - INFO - Starting async job submit for node 69c1dc9d0efaf21d931035dd 2026-03-26 00:36:52,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9d0efaf21d931035dd HTTP/1.1" 200 None 2026-03-26 00:36:52,109 - lava_callback - DEBUG - Processing node 69c1dc9d0efaf21d931035dd: baseline-arm-baylibre 2026-03-26 00:36:52,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dc9d0efaf21d931035dd 2026-03-26 00:36:52,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9d0efaf21d931035dd HTTP/1.1" 200 None 2026-03-26 00:36:52,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:36:52,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc9d0efaf21d931035dd HTTP/1.1" 200 None 2026-03-26 00:36:52,168 - lava_callback - ERROR - Error processing callback for node 69c1dc9d0efaf21d931035dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:36:52,728 - lava_callback - INFO - Received callback for node 69c1dc9f0efaf21d931035e0 INFO: 172.21.0.1:33142 - "POST /node/69c1dc9f0efaf21d931035e0 HTTP/1.0" 202 Accepted 2026-03-26 00:36:52,734 - lava_callback - INFO - Starting async job submit for node 69c1dc9f0efaf21d931035e0 2026-03-26 00:36:52,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9f0efaf21d931035e0 HTTP/1.1" 200 None 2026-03-26 00:36:52,753 - lava_callback - DEBUG - Processing node 69c1dc9f0efaf21d931035e0: baseline-arm-baylibre 2026-03-26 00:36:52,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dc9f0efaf21d931035e0 2026-03-26 00:36:52,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dc9f0efaf21d931035e0 HTTP/1.1" 200 None 2026-03-26 00:36:52,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:36:52,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:52,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dc9f0efaf21d931035e0 HTTP/1.1" 200 None 2026-03-26 00:36:52,828 - lava_callback - ERROR - Error processing callback for node 69c1dc9f0efaf21d931035e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33144 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:36:58,146 - lava_callback - INFO - Received callback for node 69c1dca70efaf21d931035f7 INFO: 172.21.0.1:33146 - "POST /node/69c1dca70efaf21d931035f7 HTTP/1.0" 202 Accepted 2026-03-26 00:36:58,151 - lava_callback - INFO - Starting async job submit for node 69c1dca70efaf21d931035f7 2026-03-26 00:36:58,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:58,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca70efaf21d931035f7 HTTP/1.1" 200 None 2026-03-26 00:36:58,167 - lava_callback - DEBUG - Processing node 69c1dca70efaf21d931035f7: baseline-arm-clabbe 2026-03-26 00:36:58,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dca70efaf21d931035f7 2026-03-26 00:36:58,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:58,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dca70efaf21d931035f7 HTTP/1.1" 200 None 2026-03-26 00:36:58,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:58,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:36:58,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:36:58,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dca70efaf21d931035f7 HTTP/1.1" 200 None 2026-03-26 00:36:58,232 - lava_callback - ERROR - Error processing callback for node 69c1dca70efaf21d931035f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39480 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:37:18,097 - lava_callback - INFO - Received callback for node 69c1dcab0efaf21d93103604 INFO: 172.21.0.1:39484 - "POST /node/69c1dcab0efaf21d93103604 HTTP/1.0" 202 Accepted 2026-03-26 00:37:18,106 - lava_callback - INFO - Starting async job submit for node 69c1dcab0efaf21d93103604 2026-03-26 00:37:18,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcab0efaf21d93103604 HTTP/1.1" 200 None 2026-03-26 00:37:18,139 - lava_callback - DEBUG - Processing node 69c1dcab0efaf21d93103604: baseline-arm-clabbe 2026-03-26 00:37:18,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dcab0efaf21d93103604 2026-03-26 00:37:18,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcab0efaf21d93103604 HTTP/1.1" 200 None 2026-03-26 00:37:18,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:37:18,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcab0efaf21d93103604 HTTP/1.1" 200 None 2026-03-26 00:37:18,249 - lava_callback - ERROR - Error processing callback for node 69c1dcab0efaf21d93103604: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:37:18,798 - lava_callback - INFO - Received callback for node 69c1dcad0efaf21d93103607 INFO: 172.21.0.1:39500 - "POST /node/69c1dcad0efaf21d93103607 HTTP/1.0" 202 Accepted 2026-03-26 00:37:18,807 - lava_callback - INFO - Starting async job submit for node 69c1dcad0efaf21d93103607 2026-03-26 00:37:18,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcad0efaf21d93103607 HTTP/1.1" 200 None 2026-03-26 00:37:18,860 - lava_callback - DEBUG - Processing node 69c1dcad0efaf21d93103607: baseline-arm-clabbe 2026-03-26 00:37:18,860 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dcad0efaf21d93103607 2026-03-26 00:37:18,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcad0efaf21d93103607 HTTP/1.1" 200 None 2026-03-26 00:37:18,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:37:18,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:18,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcad0efaf21d93103607 HTTP/1.1" 200 None 2026-03-26 00:37:18,981 - lava_callback - ERROR - Error processing callback for node 69c1dcad0efaf21d93103607: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:37:19,502 - lava_callback - INFO - Received callback for node 69c1dcae0efaf21d9310360a INFO: 172.21.0.1:39502 - "POST /node/69c1dcae0efaf21d9310360a HTTP/1.0" 202 Accepted 2026-03-26 00:37:19,513 - lava_callback - INFO - Starting async job submit for node 69c1dcae0efaf21d9310360a 2026-03-26 00:37:19,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:19,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcae0efaf21d9310360a HTTP/1.1" 200 None 2026-03-26 00:37:19,537 - lava_callback - DEBUG - Processing node 69c1dcae0efaf21d9310360a: baseline-arm-clabbe 2026-03-26 00:37:19,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dcae0efaf21d9310360a 2026-03-26 00:37:19,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:19,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcae0efaf21d9310360a HTTP/1.1" 200 None 2026-03-26 00:37:19,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:19,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:37:19,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:19,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcae0efaf21d9310360a HTTP/1.1" 200 None 2026-03-26 00:37:19,657 - lava_callback - ERROR - Error processing callback for node 69c1dcae0efaf21d9310360a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:37:20,198 - lava_callback - INFO - Received callback for node 69c1dcb10efaf21d93103613 INFO: 172.21.0.1:43560 - "POST /node/69c1dcb10efaf21d93103613 HTTP/1.0" 202 Accepted 2026-03-26 00:37:20,209 - lava_callback - INFO - Starting async job submit for node 69c1dcb10efaf21d93103613 2026-03-26 00:37:20,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:20,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb10efaf21d93103613 HTTP/1.1" 200 None 2026-03-26 00:37:20,264 - lava_callback - DEBUG - Processing node 69c1dcb10efaf21d93103613: baseline-arm-clabbe 2026-03-26 00:37:20,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dcb10efaf21d93103613 2026-03-26 00:37:20,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:20,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dcb10efaf21d93103613 HTTP/1.1" 200 None 2026-03-26 00:37:20,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:20,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 00:37:20,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:37:20,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dcb10efaf21d93103613 HTTP/1.1" 200 None 2026-03-26 00:37:20,370 - lava_callback - ERROR - Error processing callback for node 69c1dcb10efaf21d93103613: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37418 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:40:31,684 - lava_callback - INFO - Received callback for node 69c48070590270ad7b6d9091 INFO: 172.21.0.1:36056 - "POST /node/69c48070590270ad7b6d9091 HTTP/1.0" 202 Accepted 2026-03-26 00:40:31,690 - lava_callback - INFO - Starting async job submit for node 69c48070590270ad7b6d9091 2026-03-26 00:40:31,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:31,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48070590270ad7b6d9091 HTTP/1.1" 200 None 2026-03-26 00:40:31,708 - lava_callback - DEBUG - Processing node 69c48070590270ad7b6d9091: baseline-arm 2026-03-26 00:40:31,713 - lava_callback - WARNING - Failed to upload log 2026-03-26 00:40:31,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:31,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:31,831 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c48070590270ad7b6d9091/lava-logs.yaml 2026-03-26 00:40:31,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:32,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:32,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c48070590270ad7b6d9091/lava_callback.json.gz 2026-03-26 00:40:32,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:32,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48070590270ad7b6d9091 HTTP/1.1" 200 None 2026-03-26 00:40:32,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:32,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:40:32,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:32,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48070590270ad7b6d9091 HTTP/1.1" 200 None 2026-03-26 00:40:32,108 - lava_callback - INFO - Completed processing callback for node 69c48070590270ad7b6d9091 INFO: 172.21.0.1:36064 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:40:38,324 - lava_callback - INFO - Received callback for node 69c48065590270ad7b6d907d INFO: 172.21.0.1:36070 - "POST /node/69c48065590270ad7b6d907d HTTP/1.0" 202 Accepted 2026-03-26 00:40:38,330 - lava_callback - INFO - Starting async job submit for node 69c48065590270ad7b6d907d 2026-03-26 00:40:38,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:38,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48065590270ad7b6d907d HTTP/1.1" 200 None 2026-03-26 00:40:38,363 - lava_callback - DEBUG - Processing node 69c48065590270ad7b6d907d: baseline-x86 2026-03-26 00:40:38,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:38,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:38,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c48065590270ad7b6d907d/log.txt.gz 2026-03-26 00:40:38,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:39,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:39,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c48065590270ad7b6d907d/lava-logs.yaml 2026-03-26 00:40:39,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:39,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:39,176 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c48065590270ad7b6d907d/lava_callback.json.gz 2026-03-26 00:40:39,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:39,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48065590270ad7b6d907d HTTP/1.1" 200 None 2026-03-26 00:40:39,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:39,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b5e590270ad7b6d69bd HTTP/1.1" 200 None 2026-03-26 00:40:39,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:39,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48065590270ad7b6d907d HTTP/1.1" 200 None 2026-03-26 00:40:39,254 - lava_callback - INFO - Completed processing callback for node 69c48065590270ad7b6d907d 2026-03-26 00:40:40,171 - lava_callback - INFO - Received callback for node 69c4806f590270ad7b6d908e INFO: 172.21.0.1:56770 - "POST /node/69c4806f590270ad7b6d908e HTTP/1.0" 202 Accepted 2026-03-26 00:40:40,177 - lava_callback - INFO - Starting async job submit for node 69c4806f590270ad7b6d908e 2026-03-26 00:40:40,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:40,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4806f590270ad7b6d908e HTTP/1.1" 200 None 2026-03-26 00:40:40,210 - lava_callback - DEBUG - Processing node 69c4806f590270ad7b6d908e: baseline-arm 2026-03-26 00:40:40,215 - lava_callback - WARNING - Failed to upload log 2026-03-26 00:40:40,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:40,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:40,439 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4806f590270ad7b6d908e/lava-logs.yaml 2026-03-26 00:40:40,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:40,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:40,656 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4806f590270ad7b6d908e/lava_callback.json.gz 2026-03-26 00:40:40,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:40,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4806f590270ad7b6d908e HTTP/1.1" 200 None 2026-03-26 00:40:40,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:40,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:40:40,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:40,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4806f590270ad7b6d908e HTTP/1.1" 200 None 2026-03-26 00:40:40,729 - lava_callback - INFO - Completed processing callback for node 69c4806f590270ad7b6d908e INFO: 172.21.0.1:56784 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:40:47,238 - lava_callback - INFO - Received callback for node 69c48063590270ad7b6d9073 INFO: 172.21.0.1:56798 - "POST /node/69c48063590270ad7b6d9073 HTTP/1.0" 202 Accepted 2026-03-26 00:40:47,248 - lava_callback - INFO - Starting async job submit for node 69c48063590270ad7b6d9073 2026-03-26 00:40:47,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:47,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48063590270ad7b6d9073 HTTP/1.1" 200 None 2026-03-26 00:40:47,317 - lava_callback - DEBUG - Processing node 69c48063590270ad7b6d9073: baseline-x86 2026-03-26 00:40:47,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:47,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:47,663 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c48063590270ad7b6d9073/log.txt.gz 2026-03-26 00:40:47,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:47,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:47,975 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c48063590270ad7b6d9073/lava-logs.yaml 2026-03-26 00:40:47,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:40:48,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:40:48,144 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c48063590270ad7b6d9073/lava_callback.json.gz 2026-03-26 00:40:48,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:48,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48063590270ad7b6d9073 HTTP/1.1" 200 None 2026-03-26 00:40:48,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:48,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b5e590270ad7b6d69bd HTTP/1.1" 200 None 2026-03-26 00:40:48,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:48,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48063590270ad7b6d9073 HTTP/1.1" 200 None 2026-03-26 00:40:48,292 - lava_callback - INFO - Completed processing callback for node 69c48063590270ad7b6d9073 INFO: 172.21.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:40:59,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:40:59,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 00:41:05,731 - lava_callback - INFO - Received callback for node 69c48073590270ad7b6d909a INFO: 172.21.0.1:46382 - "POST /node/69c48073590270ad7b6d909a HTTP/1.0" 202 Accepted 2026-03-26 00:41:05,777 - lava_callback - INFO - Starting async job submit for node 69c48073590270ad7b6d909a 2026-03-26 00:41:05,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:05,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48073590270ad7b6d909a HTTP/1.1" 200 None 2026-03-26 00:41:05,810 - lava_callback - DEBUG - Processing node 69c48073590270ad7b6d909a: baseline-arm 2026-03-26 00:41:05,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:06,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:06,096 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c48073590270ad7b6d909a/log.txt.gz 2026-03-26 00:41:06,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:06,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:06,397 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c48073590270ad7b6d909a/lava-logs.yaml 2026-03-26 00:41:06,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:06,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:06,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c48073590270ad7b6d909a/lava_callback.json.gz 2026-03-26 00:41:06,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:41:06,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48073590270ad7b6d909a HTTP/1.1" 200 None 2026-03-26 00:41:06,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:06,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:41:06,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:06,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48073590270ad7b6d909a HTTP/1.1" 200 None 2026-03-26 00:41:06,583 - lava_callback - INFO - Completed processing callback for node 69c48073590270ad7b6d909a INFO: 172.21.0.1:58678 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 00:41:15,290 - lava_callback - INFO - Received callback for node 69c48071590270ad7b6d9097 INFO: 172.21.0.1:58690 - "POST /node/69c48071590270ad7b6d9097 HTTP/1.0" 202 Accepted 2026-03-26 00:41:15,296 - lava_callback - INFO - Starting async job submit for node 69c48071590270ad7b6d9097 2026-03-26 00:41:15,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:15,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48071590270ad7b6d9097 HTTP/1.1" 200 None 2026-03-26 00:41:15,333 - lava_callback - DEBUG - Processing node 69c48071590270ad7b6d9097: baseline-arm 2026-03-26 00:41:15,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:15,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:15,683 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c48071590270ad7b6d9097/log.txt.gz 2026-03-26 00:41:15,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:16,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:16,033 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c48071590270ad7b6d9097/lava-logs.yaml 2026-03-26 00:41:16,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:16,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:16,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c48071590270ad7b6d9097/lava_callback.json.gz 2026-03-26 00:41:16,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:16,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48071590270ad7b6d9097 HTTP/1.1" 200 None 2026-03-26 00:41:16,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:16,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:41:16,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:16,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48071590270ad7b6d9097 HTTP/1.1" 200 None 2026-03-26 00:41:16,282 - lava_callback - INFO - Completed processing callback for node 69c48071590270ad7b6d9097 INFO: 172.21.0.1:58692 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:41:25,001 - lava_callback - INFO - Received callback for node 69c48064590270ad7b6d907a INFO: 172.21.0.1:46180 - "POST /node/69c48064590270ad7b6d907a HTTP/1.0" 202 Accepted 2026-03-26 00:41:25,008 - lava_callback - INFO - Starting async job submit for node 69c48064590270ad7b6d907a 2026-03-26 00:41:25,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:25,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48064590270ad7b6d907a HTTP/1.1" 200 None 2026-03-26 00:41:25,055 - lava_callback - DEBUG - Processing node 69c48064590270ad7b6d907a: baseline-x86 2026-03-26 00:41:25,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:25,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:25,572 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c48064590270ad7b6d907a/log.txt.gz 2026-03-26 00:41:25,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:25,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:25,962 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c48064590270ad7b6d907a/lava-logs.yaml 2026-03-26 00:41:25,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:26,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:26,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c48064590270ad7b6d907a/lava_callback.json.gz 2026-03-26 00:41:26,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:26,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48064590270ad7b6d907a HTTP/1.1" 200 None 2026-03-26 00:41:26,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:26,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b5e590270ad7b6d69bd HTTP/1.1" 200 None 2026-03-26 00:41:26,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:26,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48064590270ad7b6d907a HTTP/1.1" 200 None 2026-03-26 00:41:26,258 - lava_callback - INFO - Completed processing callback for node 69c48064590270ad7b6d907a INFO: 172.21.0.1:46184 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:41:29,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:29,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49740 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:41:43,544 - lava_callback - INFO - Received callback for node 69c4806f590270ad7b6d908b INFO: 172.21.0.1:49316 - "POST /node/69c4806f590270ad7b6d908b HTTP/1.0" 202 Accepted 2026-03-26 00:41:43,550 - lava_callback - INFO - Starting async job submit for node 69c4806f590270ad7b6d908b 2026-03-26 00:41:43,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:43,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4806f590270ad7b6d908b HTTP/1.1" 200 None 2026-03-26 00:41:43,586 - lava_callback - DEBUG - Processing node 69c4806f590270ad7b6d908b: baseline-arm 2026-03-26 00:41:43,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:43,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:43,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4806f590270ad7b6d908b/log.txt.gz 2026-03-26 00:41:44,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:44,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:44,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4806f590270ad7b6d908b/lava-logs.yaml 2026-03-26 00:41:44,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:44,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:44,359 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4806f590270ad7b6d908b/lava_callback.json.gz 2026-03-26 00:41:44,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:44,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4806f590270ad7b6d908b HTTP/1.1" 200 None 2026-03-26 00:41:44,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:44,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:41:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:44,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4806f590270ad7b6d908b HTTP/1.1" 200 None 2026-03-26 00:41:44,440 - lava_callback - INFO - Completed processing callback for node 69c4806f590270ad7b6d908b INFO: 172.21.0.1:49324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:41:58,002 - lava_callback - INFO - Received callback for node 69c48082590270ad7b6d90da INFO: 172.21.0.1:43644 - "POST /node/69c48082590270ad7b6d90da HTTP/1.0" 202 Accepted 2026-03-26 00:41:58,015 - lava_callback - INFO - Starting async job submit for node 69c48082590270ad7b6d90da 2026-03-26 00:41:58,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:58,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48082590270ad7b6d90da HTTP/1.1" 200 None 2026-03-26 00:41:58,072 - lava_callback - DEBUG - Processing node 69c48082590270ad7b6d90da: baseline-arm-broonie 2026-03-26 00:41:58,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:58,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:58,434 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48082590270ad7b6d90da/log.txt.gz 2026-03-26 00:41:58,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:58,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:58,746 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48082590270ad7b6d90da/lava-logs.yaml 2026-03-26 00:41:58,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:41:58,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:41:58,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48082590270ad7b6d90da/lava_callback.json.gz 2026-03-26 00:41:58,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:58,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48082590270ad7b6d90da HTTP/1.1" 200 None 2026-03-26 00:41:58,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:58,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:41:58,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:58,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48082590270ad7b6d90da HTTP/1.1" 200 None 2026-03-26 00:41:58,972 - lava_callback - INFO - Completed processing callback for node 69c48082590270ad7b6d90da 2026-03-26 00:41:59,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:41:59,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 00:42:01,788 - lava_callback - INFO - Received callback for node 69c48096590270ad7b6d9126 INFO: 172.21.0.1:51868 - "POST /node/69c48096590270ad7b6d9126 HTTP/1.0" 202 Accepted 2026-03-26 00:42:01,837 - lava_callback - INFO - Starting async job submit for node 69c48096590270ad7b6d9126 2026-03-26 00:42:01,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:01,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48096590270ad7b6d9126 HTTP/1.1" 200 None 2026-03-26 00:42:01,875 - lava_callback - DEBUG - Processing node 69c48096590270ad7b6d9126: baseline-arm-pengutronix 2026-03-26 00:42:01,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:02,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:02,097 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48096590270ad7b6d9126/log.txt.gz 2026-03-26 00:42:02,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:02,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:02,333 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48096590270ad7b6d9126/lava-logs.yaml 2026-03-26 00:42:02,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:02,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:02,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48096590270ad7b6d9126/lava_callback.json.gz 2026-03-26 00:42:02,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:02,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48096590270ad7b6d9126 HTTP/1.1" 200 None 2026-03-26 00:42:02,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:02,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:42:02,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:02,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48096590270ad7b6d9126 HTTP/1.1" 200 None 2026-03-26 00:42:02,550 - lava_callback - INFO - Completed processing callback for node 69c48096590270ad7b6d9126 INFO: 172.21.0.1:51882 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:42:06,972 - lava_callback - INFO - Received callback for node 69c48094590270ad7b6d9120 INFO: 172.21.0.1:51886 - "POST /node/69c48094590270ad7b6d9120 HTTP/1.0" 202 Accepted 2026-03-26 00:42:06,978 - lava_callback - INFO - Starting async job submit for node 69c48094590270ad7b6d9120 2026-03-26 00:42:06,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:07,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48094590270ad7b6d9120 HTTP/1.1" 200 None 2026-03-26 00:42:07,014 - lava_callback - DEBUG - Processing node 69c48094590270ad7b6d9120: baseline-arm-pengutronix 2026-03-26 00:42:07,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:07,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:07,293 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48094590270ad7b6d9120/log.txt.gz 2026-03-26 00:42:07,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:07,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:07,553 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48094590270ad7b6d9120/lava-logs.yaml 2026-03-26 00:42:07,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:07,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:07,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48094590270ad7b6d9120/lava_callback.json.gz 2026-03-26 00:42:07,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:07,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48094590270ad7b6d9120 HTTP/1.1" 200 None 2026-03-26 00:42:07,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:07,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:42:07,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:07,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48094590270ad7b6d9120 HTTP/1.1" 200 None 2026-03-26 00:42:07,767 - lava_callback - INFO - Completed processing callback for node 69c48094590270ad7b6d9120 INFO: 172.21.0.1:49782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49796 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:42:18,202 - lava_callback - INFO - Received callback for node 69c48081590270ad7b6d90d7 INFO: 172.21.0.1:49808 - "POST /node/69c48081590270ad7b6d90d7 HTTP/1.0" 202 Accepted 2026-03-26 00:42:18,208 - lava_callback - INFO - Starting async job submit for node 69c48081590270ad7b6d90d7 2026-03-26 00:42:18,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:18,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48081590270ad7b6d90d7 HTTP/1.1" 200 None 2026-03-26 00:42:18,245 - lava_callback - DEBUG - Processing node 69c48081590270ad7b6d90d7: baseline-arm-broonie 2026-03-26 00:42:18,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:18,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:18,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48081590270ad7b6d90d7/log.txt.gz 2026-03-26 00:42:18,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:19,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:19,050 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48081590270ad7b6d90d7/lava-logs.yaml 2026-03-26 00:42:19,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:19,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:19,225 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48081590270ad7b6d90d7/lava_callback.json.gz 2026-03-26 00:42:19,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:19,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48081590270ad7b6d90d7 HTTP/1.1" 200 None 2026-03-26 00:42:19,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:19,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:42:19,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:19,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48081590270ad7b6d90d7 HTTP/1.1" 200 None 2026-03-26 00:42:19,312 - lava_callback - INFO - Completed processing callback for node 69c48081590270ad7b6d90d7 INFO: 172.21.0.1:45996 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:42:26,582 - lava_callback - INFO - Received callback for node 69c48088590270ad7b6d90f0 INFO: 172.21.0.1:46008 - "POST /node/69c48088590270ad7b6d90f0 HTTP/1.0" 202 Accepted 2026-03-26 00:42:26,588 - lava_callback - INFO - Starting async job submit for node 69c48088590270ad7b6d90f0 2026-03-26 00:42:26,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:26,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48088590270ad7b6d90f0 HTTP/1.1" 200 None 2026-03-26 00:42:26,625 - lava_callback - DEBUG - Processing node 69c48088590270ad7b6d90f0: baseline-arm-clabbe 2026-03-26 00:42:26,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:26,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:26,955 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c48088590270ad7b6d90f0/log.txt.gz 2026-03-26 00:42:27,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:27,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:27,264 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c48088590270ad7b6d90f0/lava-logs.yaml 2026-03-26 00:42:27,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:27,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:27,440 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c48088590270ad7b6d90f0/lava_callback.json.gz 2026-03-26 00:42:27,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:27,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48088590270ad7b6d90f0 HTTP/1.1" 200 None 2026-03-26 00:42:27,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:27,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:42:27,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:27,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48088590270ad7b6d90f0 HTTP/1.1" 200 None 2026-03-26 00:42:27,539 - lava_callback - INFO - Completed processing callback for node 69c48088590270ad7b6d90f0 2026-03-26 00:42:29,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:29,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 00:42:30,228 - lava_callback - INFO - Received callback for node 69c48084590270ad7b6d90dd INFO: 172.21.0.1:33762 - "POST /node/69c48084590270ad7b6d90dd HTTP/1.0" 202 Accepted 2026-03-26 00:42:30,234 - lava_callback - INFO - Starting async job submit for node 69c48084590270ad7b6d90dd 2026-03-26 00:42:30,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:30,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48084590270ad7b6d90dd HTTP/1.1" 200 None 2026-03-26 00:42:30,270 - lava_callback - DEBUG - Processing node 69c48084590270ad7b6d90dd: baseline-arm-broonie 2026-03-26 00:42:30,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:30,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:30,629 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48084590270ad7b6d90dd/log.txt.gz 2026-03-26 00:42:30,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:31,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:31,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48084590270ad7b6d90dd/lava-logs.yaml 2026-03-26 00:42:31,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:31,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:31,258 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c48084590270ad7b6d90dd/lava_callback.json.gz 2026-03-26 00:42:31,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:31,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48084590270ad7b6d90dd HTTP/1.1" 200 None 2026-03-26 00:42:31,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:31,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:42:31,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:31,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48084590270ad7b6d90dd HTTP/1.1" 200 None 2026-03-26 00:42:31,356 - lava_callback - INFO - Completed processing callback for node 69c48084590270ad7b6d90dd INFO: 172.21.0.1:33774 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:42:39,559 - lava_callback - INFO - Received callback for node 69c480ae590270ad7b6d9170 INFO: 172.21.0.1:33782 - "POST /node/69c480ae590270ad7b6d9170 HTTP/1.0" 202 Accepted 2026-03-26 00:42:39,565 - lava_callback - INFO - Starting async job submit for node 69c480ae590270ad7b6d9170 2026-03-26 00:42:39,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:39,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ae590270ad7b6d9170 HTTP/1.1" 200 None 2026-03-26 00:42:39,604 - lava_callback - DEBUG - Processing node 69c480ae590270ad7b6d9170: ltp-dio 2026-03-26 00:42:39,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:40,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:40,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c480ae590270ad7b6d9170/log.txt.gz 2026-03-26 00:42:40,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:40,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:40,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c480ae590270ad7b6d9170/lava-logs.yaml 2026-03-26 00:42:40,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:40,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:40,673 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c480ae590270ad7b6d9170/lava_callback.json.gz 2026-03-26 00:42:40,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:40,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ae590270ad7b6d9170 HTTP/1.1" 200 None 2026-03-26 00:42:40,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:40,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:42:40,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:40,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480ae590270ad7b6d9170 HTTP/1.1" 200 None 2026-03-26 00:42:40,764 - lava_callback - INFO - Completed processing callback for node 69c480ae590270ad7b6d9170 INFO: 172.21.0.1:58478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36384 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:42:52,211 - lava_callback - INFO - Received callback for node 69c4807f590270ad7b6d90d1 INFO: 172.21.0.1:36386 - "POST /node/69c4807f590270ad7b6d90d1 HTTP/1.0" 202 Accepted 2026-03-26 00:42:52,217 - lava_callback - INFO - Starting async job submit for node 69c4807f590270ad7b6d90d1 2026-03-26 00:42:52,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:52,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807f590270ad7b6d90d1 HTTP/1.1" 200 None 2026-03-26 00:42:52,255 - lava_callback - DEBUG - Processing node 69c4807f590270ad7b6d90d1: baseline-arm-broonie 2026-03-26 00:42:52,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:52,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:52,602 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4807f590270ad7b6d90d1/log.txt.gz 2026-03-26 00:42:52,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:52,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:52,895 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4807f590270ad7b6d90d1/lava-logs.yaml 2026-03-26 00:42:52,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:42:53,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:42:53,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c4807f590270ad7b6d90d1/lava_callback.json.gz 2026-03-26 00:42:53,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:53,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807f590270ad7b6d90d1 HTTP/1.1" 200 None 2026-03-26 00:42:53,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:53,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:42:53,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:53,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4807f590270ad7b6d90d1 HTTP/1.1" 200 None 2026-03-26 00:42:53,145 - lava_callback - INFO - Completed processing callback for node 69c4807f590270ad7b6d90d1 INFO: 172.21.0.1:36398 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:42:59,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:42:59,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:43:06,605 - lava_callback - INFO - Received callback for node 69c48093590270ad7b6d911d INFO: 172.21.0.1:60314 - "POST /node/69c48093590270ad7b6d911d HTTP/1.0" 202 Accepted 2026-03-26 00:43:06,611 - lava_callback - INFO - Starting async job submit for node 69c48093590270ad7b6d911d 2026-03-26 00:43:06,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:43:06,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48093590270ad7b6d911d HTTP/1.1" 200 None 2026-03-26 00:43:06,648 - lava_callback - DEBUG - Processing node 69c48093590270ad7b6d911d: baseline-arm-pengutronix 2026-03-26 00:43:06,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:43:06,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:43:06,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48093590270ad7b6d911d/log.txt.gz 2026-03-26 00:43:07,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:43:07,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:43:07,213 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48093590270ad7b6d911d/lava-logs.yaml 2026-03-26 00:43:07,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:43:07,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:43:07,349 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48093590270ad7b6d911d/lava_callback.json.gz 2026-03-26 00:43:07,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:43:07,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48093590270ad7b6d911d HTTP/1.1" 200 None 2026-03-26 00:43:07,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:43:07,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:43:07,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:43:07,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48093590270ad7b6d911d HTTP/1.1" 200 None 2026-03-26 00:43:07,507 - lava_callback - INFO - Completed processing callback for node 69c48093590270ad7b6d911d INFO: 172.21.0.1:50992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:43:29,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:43:29,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53504 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:44:13,603 - lava_callback - INFO - Received callback for node 69c48089590270ad7b6d90f8 INFO: 172.21.0.1:48168 - "POST /node/69c48089590270ad7b6d90f8 HTTP/1.0" 202 Accepted 2026-03-26 00:44:13,609 - lava_callback - INFO - Starting async job submit for node 69c48089590270ad7b6d90f8 2026-03-26 00:44:13,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:44:13,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48089590270ad7b6d90f8 HTTP/1.1" 200 None 2026-03-26 00:44:13,646 - lava_callback - DEBUG - Processing node 69c48089590270ad7b6d90f8: baseline-arm-clabbe 2026-03-26 00:44:13,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:44:13,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:44:13,989 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c48089590270ad7b6d90f8/log.txt.gz 2026-03-26 00:44:14,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:44:14,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:44:14,333 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c48089590270ad7b6d90f8/lava-logs.yaml 2026-03-26 00:44:14,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:44:14,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:44:14,514 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c48089590270ad7b6d90f8/lava_callback.json.gz 2026-03-26 00:44:14,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:44:14,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48089590270ad7b6d90f8 HTTP/1.1" 200 None 2026-03-26 00:44:14,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:44:14,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:44:14,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:44:14,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48089590270ad7b6d90f8 HTTP/1.1" 200 None 2026-03-26 00:44:14,612 - lava_callback - INFO - Completed processing callback for node 69c48089590270ad7b6d90f8 INFO: 172.21.0.1:48176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57872 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:44:29,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:44:29,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53464 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:45:10,104 - lava_callback - INFO - Received callback for node 69c480a7590270ad7b6d915e INFO: 172.21.0.1:40926 - "POST /node/69c480a7590270ad7b6d915e HTTP/1.0" 202 Accepted 2026-03-26 00:45:10,110 - lava_callback - INFO - Starting async job submit for node 69c480a7590270ad7b6d915e 2026-03-26 00:45:10,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:10,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480a7590270ad7b6d915e HTTP/1.1" 200 None 2026-03-26 00:45:10,155 - lava_callback - DEBUG - Processing node 69c480a7590270ad7b6d915e: ltp-cap_bounds 2026-03-26 00:45:10,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:10,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:10,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c480a7590270ad7b6d915e/log.txt.gz 2026-03-26 00:45:10,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:11,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:11,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c480a7590270ad7b6d915e/lava-logs.yaml 2026-03-26 00:45:11,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:11,230 - lava_callback - INFO - Received callback for node 69c4818b590270ad7b6d9244 2026-03-26 00:45:11,237 - lava_callback - INFO - Starting async job submit for node 69c4818b590270ad7b6d9244 INFO: 172.21.0.1:40932 - "POST /node/69c4818b590270ad7b6d9244 HTTP/1.0" 202 Accepted 2026-03-26 00:45:11,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:11,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4818b590270ad7b6d9244 HTTP/1.1" 200 None 2026-03-26 00:45:11,258 - lava_callback - DEBUG - Processing node 69c4818b590270ad7b6d9244: baseline-arm 2026-03-26 00:45:11,263 - lava_callback - WARNING - Failed to upload log 2026-03-26 00:45:11,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:11,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:11,328 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c480a7590270ad7b6d915e/lava_callback.json.gz 2026-03-26 00:45:11,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:11,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480a7590270ad7b6d915e HTTP/1.1" 200 None 2026-03-26 00:45:11,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:11,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:11,382 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4818b590270ad7b6d9244/lava-logs.yaml 2026-03-26 00:45:11,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:11,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:45:11,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:11,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480a7590270ad7b6d915e HTTP/1.1" 200 None 2026-03-26 00:45:11,431 - lava_callback - INFO - Completed processing callback for node 69c480a7590270ad7b6d915e 2026-03-26 00:45:11,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:11,596 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4818b590270ad7b6d9244/lava_callback.json.gz 2026-03-26 00:45:11,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:11,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4818b590270ad7b6d9244 HTTP/1.1" 200 None 2026-03-26 00:45:11,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:11,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:45:11,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:11,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4818b590270ad7b6d9244 HTTP/1.1" 200 None 2026-03-26 00:45:11,685 - lava_callback - INFO - Completed processing callback for node 69c4818b590270ad7b6d9244 INFO: 172.21.0.1:40948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37140 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:45:29,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:29,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 00:45:30,256 - lava_callback - INFO - Received callback for node 69c480a8590270ad7b6d9161 INFO: 172.21.0.1:54758 - "POST /node/69c480a8590270ad7b6d9161 HTTP/1.0" 202 Accepted 2026-03-26 00:45:30,262 - lava_callback - INFO - Starting async job submit for node 69c480a8590270ad7b6d9161 2026-03-26 00:45:30,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:30,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480a8590270ad7b6d9161 HTTP/1.1" 200 None 2026-03-26 00:45:30,313 - lava_callback - DEBUG - Processing node 69c480a8590270ad7b6d9161: ltp-containers 2026-03-26 00:45:30,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:30,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:30,882 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c480a8590270ad7b6d9161/log.txt.gz 2026-03-26 00:45:31,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:31,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:31,326 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c480a8590270ad7b6d9161/lava-logs.yaml 2026-03-26 00:45:31,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:31,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:31,527 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c480a8590270ad7b6d9161/lava_callback.json.gz 2026-03-26 00:45:31,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:31,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480a8590270ad7b6d9161 HTTP/1.1" 200 None 2026-03-26 00:45:31,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:31,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:45:31,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:31,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480a8590270ad7b6d9161 HTTP/1.1" 200 None 2026-03-26 00:45:31,613 - lava_callback - INFO - Completed processing callback for node 69c480a8590270ad7b6d9161 INFO: 172.21.0.1:54762 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:45:39,663 - lava_callback - INFO - Received callback for node 69c480aa590270ad7b6d9164 INFO: 172.21.0.1:54766 - "POST /node/69c480aa590270ad7b6d9164 HTTP/1.0" 202 Accepted 2026-03-26 00:45:39,674 - lava_callback - INFO - Starting async job submit for node 69c480aa590270ad7b6d9164 2026-03-26 00:45:39,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:39,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480aa590270ad7b6d9164 HTTP/1.1" 200 None 2026-03-26 00:45:39,716 - lava_callback - DEBUG - Processing node 69c480aa590270ad7b6d9164: ltp-controllers 2026-03-26 00:45:40,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:40,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:40,291 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c480aa590270ad7b6d9164/log.txt.gz 2026-03-26 00:45:40,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:40,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:40,752 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c480aa590270ad7b6d9164/lava-logs.yaml 2026-03-26 00:45:40,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:41,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:41,110 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c480aa590270ad7b6d9164/lava_callback.json.gz 2026-03-26 00:45:41,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:41,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480aa590270ad7b6d9164 HTTP/1.1" 200 None 2026-03-26 00:45:41,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:41,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:45:41,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:41,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480aa590270ad7b6d9164 HTTP/1.1" 200 None 2026-03-26 00:45:41,191 - lava_callback - INFO - Completed processing callback for node 69c480aa590270ad7b6d9164 2026-03-26 00:45:43,372 - lava_callback - INFO - Received callback for node 69c4818c590270ad7b6d9247 INFO: 172.21.0.1:32842 - "POST /node/69c4818c590270ad7b6d9247 HTTP/1.0" 202 Accepted 2026-03-26 00:45:43,422 - lava_callback - INFO - Starting async job submit for node 69c4818c590270ad7b6d9247 2026-03-26 00:45:43,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:43,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4818c590270ad7b6d9247 HTTP/1.1" 200 None 2026-03-26 00:45:43,463 - lava_callback - DEBUG - Processing node 69c4818c590270ad7b6d9247: baseline-arm 2026-03-26 00:45:43,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:43,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:43,727 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c4818c590270ad7b6d9247/log.txt.gz 2026-03-26 00:45:43,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:43,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:43,981 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4818c590270ad7b6d9247/lava-logs.yaml 2026-03-26 00:45:43,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:44,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:44,100 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4818c590270ad7b6d9247/lava_callback.json.gz 2026-03-26 00:45:44,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:44,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4818c590270ad7b6d9247 HTTP/1.1" 200 None 2026-03-26 00:45:44,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:44,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:45:44,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:44,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4818c590270ad7b6d9247 HTTP/1.1" 200 None 2026-03-26 00:45:44,184 - lava_callback - INFO - Completed processing callback for node 69c4818c590270ad7b6d9247 INFO: 172.21.0.1:32846 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:45:50,635 - lava_callback - INFO - Received callback for node 69c480ba590270ad7b6d91a8 INFO: 172.21.0.1:44476 - "POST /node/69c480ba590270ad7b6d91a8 HTTP/1.0" 202 Accepted 2026-03-26 00:45:50,642 - lava_callback - INFO - Starting async job submit for node 69c480ba590270ad7b6d91a8 2026-03-26 00:45:50,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:50,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ba590270ad7b6d91a8 HTTP/1.1" 200 None 2026-03-26 00:45:50,683 - lava_callback - DEBUG - Processing node 69c480ba590270ad7b6d91a8: ltp-syscalls 2026-03-26 00:45:51,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:51,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:51,515 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c480ba590270ad7b6d91a8/log.txt.gz INFO: 172.21.0.1:44484 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:45:51,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:52,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:52,223 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c480ba590270ad7b6d91a8/lava-logs.yaml 2026-03-26 00:45:52,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:52,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:52,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c480ba590270ad7b6d91a8/lava_callback.json.gz 2026-03-26 00:45:52,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:52,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ba590270ad7b6d91a8 HTTP/1.1" 200 None 2026-03-26 00:45:52,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:52,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:45:52,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:52,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480ba590270ad7b6d91a8 HTTP/1.1" 200 None 2026-03-26 00:45:52,484 - lava_callback - INFO - Completed processing callback for node 69c480ba590270ad7b6d91a8 2026-03-26 00:45:52,763 - lava_callback - INFO - Received callback for node 69c4808d590270ad7b6d9105 INFO: 172.21.0.1:44486 - "POST /node/69c4808d590270ad7b6d9105 HTTP/1.0" 202 Accepted 2026-03-26 00:45:52,769 - lava_callback - INFO - Starting async job submit for node 69c4808d590270ad7b6d9105 2026-03-26 00:45:52,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:52,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808d590270ad7b6d9105 HTTP/1.1" 200 None 2026-03-26 00:45:52,812 - lava_callback - DEBUG - Processing node 69c4808d590270ad7b6d9105: baseline-arm-clabbe 2026-03-26 00:45:53,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:53,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:53,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4808d590270ad7b6d9105/log.txt.gz 2026-03-26 00:45:53,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:53,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:53,484 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4808d590270ad7b6d9105/lava-logs.yaml 2026-03-26 00:45:53,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:45:53,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:45:53,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4808d590270ad7b6d9105/lava_callback.json.gz 2026-03-26 00:45:53,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:53,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808d590270ad7b6d9105 HTTP/1.1" 200 None 2026-03-26 00:45:53,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:53,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:45:53,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:53,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808d590270ad7b6d9105 HTTP/1.1" 200 None 2026-03-26 00:45:53,699 - lava_callback - INFO - Completed processing callback for node 69c4808d590270ad7b6d9105 INFO: 172.21.0.1:44496 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:45:59,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:45:59,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:46:19,445 - lava_callback - INFO - Received callback for node 69c481d3590270ad7b6d928c INFO: 172.21.0.1:40610 - "POST /node/69c481d3590270ad7b6d928c HTTP/1.0" 202 Accepted 2026-03-26 00:46:19,452 - lava_callback - INFO - Starting async job submit for node 69c481d3590270ad7b6d928c 2026-03-26 00:46:19,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:19,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c481d3590270ad7b6d928c HTTP/1.1" 200 None 2026-03-26 00:46:19,486 - lava_callback - DEBUG - Processing node 69c481d3590270ad7b6d928c: baseline-arm 2026-03-26 00:46:19,491 - lava_callback - WARNING - Failed to upload log 2026-03-26 00:46:19,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:46:19,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:46:19,640 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c481d3590270ad7b6d928c/lava-logs.yaml 2026-03-26 00:46:19,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:46:19,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:46:19,806 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c481d3590270ad7b6d928c/lava_callback.json.gz 2026-03-26 00:46:19,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:19,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c481d3590270ad7b6d928c HTTP/1.1" 200 None 2026-03-26 00:46:19,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:19,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:46:19,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:19,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c481d3590270ad7b6d928c HTTP/1.1" 200 None 2026-03-26 00:46:19,870 - lava_callback - INFO - Completed processing callback for node 69c481d3590270ad7b6d928c INFO: 172.21.0.1:50320 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:46:29,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:29,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41516 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:46:49,363 - lava_callback - INFO - Received callback for node 69c480ac590270ad7b6d916b INFO: 172.21.0.1:41532 - "POST /node/69c480ac590270ad7b6d916b HTTP/1.0" 202 Accepted 2026-03-26 00:46:49,370 - lava_callback - INFO - Starting async job submit for node 69c480ac590270ad7b6d916b 2026-03-26 00:46:49,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:49,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ac590270ad7b6d916b HTTP/1.1" 200 None 2026-03-26 00:46:49,415 - lava_callback - DEBUG - Processing node 69c480ac590270ad7b6d916b: ltp-cve 2026-03-26 00:46:49,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:46:49,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:46:49,975 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c480ac590270ad7b6d916b/log.txt.gz 2026-03-26 00:46:50,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:46:50,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:46:50,444 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c480ac590270ad7b6d916b/lava-logs.yaml 2026-03-26 00:46:50,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:46:50,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:46:50,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c480ac590270ad7b6d916b/lava_callback.json.gz 2026-03-26 00:46:50,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:50,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ac590270ad7b6d916b HTTP/1.1" 200 None 2026-03-26 00:46:50,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:50,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:46:50,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:50,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480ac590270ad7b6d916b HTTP/1.1" 200 None 2026-03-26 00:46:50,732 - lava_callback - INFO - Completed processing callback for node 69c480ac590270ad7b6d916b INFO: 172.21.0.1:48484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48492 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:46:59,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:46:59,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:47:06,940 - lava_callback - INFO - Received callback for node 69c48063590270ad7b6d9077 INFO: 172.21.0.1:42712 - "POST /node/69c48063590270ad7b6d9077 HTTP/1.0" 202 Accepted 2026-03-26 00:47:06,946 - lava_callback - INFO - Starting async job submit for node 69c48063590270ad7b6d9077 2026-03-26 00:47:06,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:06,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48063590270ad7b6d9077 HTTP/1.1" 200 None 2026-03-26 00:47:06,984 - lava_callback - DEBUG - Processing node 69c48063590270ad7b6d9077: baseline-x86 2026-03-26 00:47:07,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:07,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:07,595 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c48063590270ad7b6d9077/log.txt.gz 2026-03-26 00:47:07,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:08,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:08,005 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c48063590270ad7b6d9077/lava-logs.yaml 2026-03-26 00:47:08,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:08,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:08,204 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c48063590270ad7b6d9077/lava_callback.json.gz 2026-03-26 00:47:08,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:08,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48063590270ad7b6d9077 HTTP/1.1" 200 None 2026-03-26 00:47:08,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:08,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b5e590270ad7b6d69bd HTTP/1.1" 200 None 2026-03-26 00:47:08,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:08,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48063590270ad7b6d9077 HTTP/1.1" 200 None 2026-03-26 00:47:08,291 - lava_callback - INFO - Completed processing callback for node 69c48063590270ad7b6d9077 INFO: 172.21.0.1:53044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53052 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:47:19,784 - lava_callback - INFO - Received callback for node 69c4808e590270ad7b6d9108 INFO: 172.21.0.1:47006 - "POST /node/69c4808e590270ad7b6d9108 HTTP/1.0" 202 Accepted 2026-03-26 00:47:19,790 - lava_callback - INFO - Starting async job submit for node 69c4808e590270ad7b6d9108 2026-03-26 00:47:19,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:19,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808e590270ad7b6d9108 HTTP/1.1" 200 None 2026-03-26 00:47:19,826 - lava_callback - DEBUG - Processing node 69c4808e590270ad7b6d9108: baseline-arm-clabbe 2026-03-26 00:47:20,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:20,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:20,186 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4808e590270ad7b6d9108/log.txt.gz 2026-03-26 00:47:20,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:20,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:20,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4808e590270ad7b6d9108/lava-logs.yaml 2026-03-26 00:47:20,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:20,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:20,619 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c4808e590270ad7b6d9108/lava_callback.json.gz 2026-03-26 00:47:20,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:20,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808e590270ad7b6d9108 HTTP/1.1" 200 None 2026-03-26 00:47:20,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:20,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:47:20,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:20,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808e590270ad7b6d9108 HTTP/1.1" 200 None 2026-03-26 00:47:20,724 - lava_callback - INFO - Completed processing callback for node 69c4808e590270ad7b6d9108 INFO: 172.21.0.1:47010 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:47:29,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:29,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 00:47:29,438 - lava_callback - INFO - Received callback for node 69c480af590270ad7b6d9173 INFO: 172.21.0.1:47024 - "POST /node/69c480af590270ad7b6d9173 HTTP/1.0" 202 Accepted 2026-03-26 00:47:29,444 - lava_callback - INFO - Starting async job submit for node 69c480af590270ad7b6d9173 2026-03-26 00:47:29,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:29,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480af590270ad7b6d9173 HTTP/1.1" 200 None 2026-03-26 00:47:29,484 - lava_callback - DEBUG - Processing node 69c480af590270ad7b6d9173: ltp-fs_bind 2026-03-26 00:47:29,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:30,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:30,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c480af590270ad7b6d9173/log.txt.gz 2026-03-26 00:47:30,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:30,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:30,480 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c480af590270ad7b6d9173/lava-logs.yaml 2026-03-26 00:47:30,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:47:30,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:47:30,653 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c480af590270ad7b6d9173/lava_callback.json.gz 2026-03-26 00:47:30,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:30,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480af590270ad7b6d9173 HTTP/1.1" 200 None 2026-03-26 00:47:30,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:30,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:47:30,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:30,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480af590270ad7b6d9173 HTTP/1.1" 200 None 2026-03-26 00:47:30,761 - lava_callback - INFO - Completed processing callback for node 69c480af590270ad7b6d9173 INFO: 172.21.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54366 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:47:59,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:47:59,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52790 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:48:58,639 - lava_callback - INFO - Received callback for node 69c1df680efaf21d9310386f INFO: 172.21.0.1:52794 - "POST /node/69c1df680efaf21d9310386f HTTP/1.0" 202 Accepted 2026-03-26 00:48:58,644 - lava_callback - INFO - Starting async job submit for node 69c1df680efaf21d9310386f 2026-03-26 00:48:58,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:48:58,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1df680efaf21d9310386f HTTP/1.1" 200 None 2026-03-26 00:48:58,661 - lava_callback - DEBUG - Processing node 69c1df680efaf21d9310386f: baseline-arm-clabbe 2026-03-26 00:48:58,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1df680efaf21d9310386f 2026-03-26 00:48:58,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:48:58,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1df680efaf21d9310386f HTTP/1.1" 200 None 2026-03-26 00:48:58,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:48:58,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 00:48:58,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:48:58,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1df680efaf21d9310386f HTTP/1.1" 200 None 2026-03-26 00:48:58,721 - lava_callback - ERROR - Error processing callback for node 69c1df680efaf21d9310386f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41870 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:49:09,389 - lava_callback - INFO - Received callback for node 69c480b0590270ad7b6d9177 INFO: 172.21.0.1:41884 - "POST /node/69c480b0590270ad7b6d9177 HTTP/1.0" 202 Accepted 2026-03-26 00:49:09,395 - lava_callback - INFO - Starting async job submit for node 69c480b0590270ad7b6d9177 2026-03-26 00:49:09,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:09,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b0590270ad7b6d9177 HTTP/1.1" 200 None 2026-03-26 00:49:09,434 - lava_callback - DEBUG - Processing node 69c480b0590270ad7b6d9177: ltp-fsx 2026-03-26 00:49:09,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:49:09,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:49:09,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c480b0590270ad7b6d9177/log.txt.gz 2026-03-26 00:49:10,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:49:10,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:49:10,369 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c480b0590270ad7b6d9177/lava-logs.yaml 2026-03-26 00:49:10,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:49:10,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:49:10,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c480b0590270ad7b6d9177/lava_callback.json.gz 2026-03-26 00:49:10,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:10,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b0590270ad7b6d9177 HTTP/1.1" 200 None 2026-03-26 00:49:10,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:10,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:49:10,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:10,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480b0590270ad7b6d9177 HTTP/1.1" 200 None 2026-03-26 00:49:10,636 - lava_callback - INFO - Completed processing callback for node 69c480b0590270ad7b6d9177 INFO: 172.21.0.1:52272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44656 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:49:29,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:29,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41506 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:49:38,620 - lava_callback - INFO - Received callback for node 69c1df8e0efaf21d93103882 INFO: 172.21.0.1:41512 - "POST /node/69c1df8e0efaf21d93103882 HTTP/1.0" 202 Accepted 2026-03-26 00:49:38,626 - lava_callback - INFO - Starting async job submit for node 69c1df8e0efaf21d93103882 2026-03-26 00:49:38,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:38,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1df8e0efaf21d93103882 HTTP/1.1" 200 None 2026-03-26 00:49:38,642 - lava_callback - DEBUG - Processing node 69c1df8e0efaf21d93103882: baseline-arm-clabbe 2026-03-26 00:49:38,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1df8e0efaf21d93103882 2026-03-26 00:49:38,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:38,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1df8e0efaf21d93103882 HTTP/1.1" 200 None 2026-03-26 00:49:38,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:38,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 00:49:38,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:38,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1df8e0efaf21d93103882 HTTP/1.1" 200 None 2026-03-26 00:49:38,705 - lava_callback - ERROR - Error processing callback for node 69c1df8e0efaf21d93103882: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34024 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:49:58,728 - lava_callback - INFO - Received callback for node 69c480b4590270ad7b6d917d INFO: 172.21.0.1:34026 - "POST /node/69c480b4590270ad7b6d917d HTTP/1.0" 202 Accepted 2026-03-26 00:49:58,734 - lava_callback - INFO - Starting async job submit for node 69c480b4590270ad7b6d917d 2026-03-26 00:49:58,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:58,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b4590270ad7b6d917d HTTP/1.1" 200 None 2026-03-26 00:49:58,776 - lava_callback - DEBUG - Processing node 69c480b4590270ad7b6d917d: ltp-ipc 2026-03-26 00:49:59,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:49:59,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:49:59,253 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c480b4590270ad7b6d917d/log.txt.gz 2026-03-26 00:49:59,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:49:59,617 - lava_callback - INFO - Received callback for node 69c48270590270ad7b6d92d7 INFO: 172.21.0.1:34030 - "POST /node/69c48270590270ad7b6d92d7 HTTP/1.0" 202 Accepted 2026-03-26 00:49:59,622 - lava_callback - INFO - Starting async job submit for node 69c48270590270ad7b6d92d7 2026-03-26 00:49:59,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:49:59,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48270590270ad7b6d92d7 HTTP/1.1" 200 None 2026-03-26 00:49:59,661 - lava_callback - DEBUG - Processing node 69c48270590270ad7b6d92d7: baseline-arm-pengutronix 2026-03-26 00:49:59,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:49:59,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:49:59,946 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48270590270ad7b6d92d7/log.txt.gz 2026-03-26 00:50:00,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:00,038 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c480b4590270ad7b6d917d/lava-logs.yaml 2026-03-26 00:50:00,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:00,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:00,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:00,236 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c480b4590270ad7b6d917d/lava_callback.json.gz 2026-03-26 00:50:00,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:00,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:00,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48270590270ad7b6d92d7/lava-logs.yaml 2026-03-26 00:50:00,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b4590270ad7b6d917d HTTP/1.1" 200 None 2026-03-26 00:50:00,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:00,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:00,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:50:00,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:00,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480b4590270ad7b6d917d HTTP/1.1" 200 None 2026-03-26 00:50:00,325 - lava_callback - INFO - Completed processing callback for node 69c480b4590270ad7b6d917d 2026-03-26 00:50:00,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:00,516 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48270590270ad7b6d92d7/lava_callback.json.gz 2026-03-26 00:50:00,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:00,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48270590270ad7b6d92d7 HTTP/1.1" 200 None 2026-03-26 00:50:00,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:00,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:50:00,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:00,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48270590270ad7b6d92d7 HTTP/1.1" 200 None 2026-03-26 00:50:00,618 - lava_callback - INFO - Completed processing callback for node 69c48270590270ad7b6d92d7 INFO: 172.21.0.1:34060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54446 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:50:18,760 - lava_callback - INFO - Received callback for node 69c1dfb70efaf21d93105e2e INFO: 172.21.0.1:54460 - "POST /node/69c1dfb70efaf21d93105e2e HTTP/1.0" 202 Accepted 2026-03-26 00:50:18,765 - lava_callback - INFO - Starting async job submit for node 69c1dfb70efaf21d93105e2e 2026-03-26 00:50:18,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:18,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dfb70efaf21d93105e2e HTTP/1.1" 200 None 2026-03-26 00:50:18,782 - lava_callback - DEBUG - Processing node 69c1dfb70efaf21d93105e2e: baseline-arm-clabbe 2026-03-26 00:50:18,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dfb70efaf21d93105e2e 2026-03-26 00:50:18,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:18,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dfb70efaf21d93105e2e HTTP/1.1" 200 None 2026-03-26 00:50:18,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:18,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 00:50:18,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:18,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dfb70efaf21d93105e2e HTTP/1.1" 200 None 2026-03-26 00:50:18,841 - lava_callback - ERROR - Error processing callback for node 69c1dfb70efaf21d93105e2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46546 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:50:29,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:29,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:50:38,996 - lava_callback - INFO - Received callback for node 69c47e4a590270ad7b6d8ff7 INFO: 172.21.0.1:56432 - "POST /node/69c47e4a590270ad7b6d8ff7 HTTP/1.0" 202 Accepted 2026-03-26 00:50:39,004 - lava_callback - INFO - Starting async job submit for node 69c47e4a590270ad7b6d8ff7 2026-03-26 00:50:39,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:39,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e4a590270ad7b6d8ff7 HTTP/1.1" 200 None 2026-03-26 00:50:39,076 - lava_callback - DEBUG - Processing node 69c47e4a590270ad7b6d8ff7: ltp-smoke 2026-03-26 00:50:39,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:39,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:39,695 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c47e4a590270ad7b6d8ff7/log.txt.gz 2026-03-26 00:50:39,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:40,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:40,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c47e4a590270ad7b6d8ff7/lava-logs.yaml 2026-03-26 00:50:40,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:40,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:40,415 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c47e4a590270ad7b6d8ff7/lava_callback.json.gz 2026-03-26 00:50:40,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:40,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e4a590270ad7b6d8ff7 HTTP/1.1" 200 None 2026-03-26 00:50:40,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:40,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b41590270ad7b6d6994 HTTP/1.1" 200 None 2026-03-26 00:50:40,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:40,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47e4a590270ad7b6d8ff7 HTTP/1.1" 200 None 2026-03-26 00:50:40,734 - lava_callback - INFO - Completed processing callback for node 69c47e4a590270ad7b6d8ff7 INFO: 172.21.0.1:45958 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:50:49,600 - lava_callback - INFO - Received callback for node 69c480b6590270ad7b6d9182 INFO: 172.21.0.1:45964 - "POST /node/69c480b6590270ad7b6d9182 HTTP/1.0" 202 Accepted 2026-03-26 00:50:49,606 - lava_callback - INFO - Starting async job submit for node 69c480b6590270ad7b6d9182 2026-03-26 00:50:49,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:49,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b6590270ad7b6d9182 HTTP/1.1" 200 None 2026-03-26 00:50:49,647 - lava_callback - DEBUG - Processing node 69c480b6590270ad7b6d9182: ltp-pty 2026-03-26 00:50:49,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:50,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:50,124 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c480b6590270ad7b6d9182/log.txt.gz 2026-03-26 00:50:50,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:50,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:50,558 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c480b6590270ad7b6d9182/lava-logs.yaml 2026-03-26 00:50:50,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:50:50,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:50:50,743 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c480b6590270ad7b6d9182/lava_callback.json.gz 2026-03-26 00:50:50,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:50,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b6590270ad7b6d9182 HTTP/1.1" 200 None 2026-03-26 00:50:50,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:50,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:50:50,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:50,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480b6590270ad7b6d9182 HTTP/1.1" 200 None 2026-03-26 00:50:50,833 - lava_callback - INFO - Completed processing callback for node 69c480b6590270ad7b6d9182 INFO: 172.21.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37616 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:50:58,726 - lava_callback - INFO - Received callback for node 69c1dfe00efaf21d93105e40 INFO: 172.21.0.1:37622 - "POST /node/69c1dfe00efaf21d93105e40 HTTP/1.0" 202 Accepted 2026-03-26 00:50:58,731 - lava_callback - INFO - Starting async job submit for node 69c1dfe00efaf21d93105e40 2026-03-26 00:50:58,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:58,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dfe00efaf21d93105e40 HTTP/1.1" 200 None 2026-03-26 00:50:58,748 - lava_callback - DEBUG - Processing node 69c1dfe00efaf21d93105e40: baseline-arm-clabbe 2026-03-26 00:50:58,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1dfe00efaf21d93105e40 2026-03-26 00:50:58,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:58,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1dfe00efaf21d93105e40 HTTP/1.1" 200 None 2026-03-26 00:50:58,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:58,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 00:50:58,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:58,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1dfe00efaf21d93105e40 HTTP/1.1" 200 None 2026-03-26 00:50:58,818 - lava_callback - ERROR - Error processing callback for node 69c1dfe00efaf21d93105e40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:50:59,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:50:59,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:51:09,318 - lava_callback - INFO - Received callback for node 69c482d4590270ad7b6d930a INFO: 172.21.0.1:56598 - "POST /node/69c482d4590270ad7b6d930a HTTP/1.0" 202 Accepted 2026-03-26 00:51:09,365 - lava_callback - INFO - Starting async job submit for node 69c482d4590270ad7b6d930a 2026-03-26 00:51:09,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:09,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c482d4590270ad7b6d930a HTTP/1.1" 200 None 2026-03-26 00:51:09,398 - lava_callback - DEBUG - Processing node 69c482d4590270ad7b6d930a: baseline-arm 2026-03-26 00:51:09,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:51:09,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:51:09,644 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c482d4590270ad7b6d930a/log.txt.gz 2026-03-26 00:51:09,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:51:09,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:51:09,919 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c482d4590270ad7b6d930a/lava-logs.yaml 2026-03-26 00:51:09,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:51:10,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:51:10,034 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c482d4590270ad7b6d930a/lava_callback.json.gz 2026-03-26 00:51:10,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:10,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c482d4590270ad7b6d930a HTTP/1.1" 200 None 2026-03-26 00:51:10,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:10,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:51:10,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:10,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c482d4590270ad7b6d930a HTTP/1.1" 200 None 2026-03-26 00:51:10,122 - lava_callback - INFO - Completed processing callback for node 69c482d4590270ad7b6d930a INFO: 172.21.0.1:49278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:51:16,820 - lava_callback - INFO - Received callback for node 69c482d5590270ad7b6d9317 INFO: 172.21.0.1:49304 - "POST /node/69c482d5590270ad7b6d9317 HTTP/1.0" 202 Accepted 2026-03-26 00:51:16,870 - lava_callback - INFO - Starting async job submit for node 69c482d5590270ad7b6d9317 2026-03-26 00:51:16,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:16,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c482d5590270ad7b6d9317 HTTP/1.1" 200 None 2026-03-26 00:51:16,902 - lava_callback - DEBUG - Processing node 69c482d5590270ad7b6d9317: baseline-arm 2026-03-26 00:51:17,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:51:17,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:51:17,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c482d5590270ad7b6d9317/log.txt.gz 2026-03-26 00:51:17,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:51:17,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:51:17,419 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c482d5590270ad7b6d9317/lava-logs.yaml 2026-03-26 00:51:17,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:51:17,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:51:17,538 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c482d5590270ad7b6d9317/lava_callback.json.gz 2026-03-26 00:51:17,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:17,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c482d5590270ad7b6d9317 HTTP/1.1" 200 None 2026-03-26 00:51:17,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:17,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:51:17,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:17,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c482d5590270ad7b6d9317 HTTP/1.1" 200 None 2026-03-26 00:51:17,613 - lava_callback - INFO - Completed processing callback for node 69c482d5590270ad7b6d9317 INFO: 172.21.0.1:50006 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:51:29,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:29,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:51:38,810 - lava_callback - INFO - Received callback for node 69c1e0080efaf21d93105e55 INFO: 172.21.0.1:56002 - "POST /node/69c1e0080efaf21d93105e55 HTTP/1.0" 202 Accepted 2026-03-26 00:51:38,819 - lava_callback - INFO - Starting async job submit for node 69c1e0080efaf21d93105e55 2026-03-26 00:51:38,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:38,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e0080efaf21d93105e55 HTTP/1.1" 200 None 2026-03-26 00:51:38,850 - lava_callback - DEBUG - Processing node 69c1e0080efaf21d93105e55: baseline-arm-clabbe 2026-03-26 00:51:38,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e0080efaf21d93105e55 2026-03-26 00:51:38,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:38,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e0080efaf21d93105e55 HTTP/1.1" 200 None 2026-03-26 00:51:38,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:38,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 00:51:38,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:51:38,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e0080efaf21d93105e55 HTTP/1.1" 200 None 2026-03-26 00:51:38,962 - lava_callback - ERROR - Error processing callback for node 69c1e0080efaf21d93105e55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41022 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:52:00,977 - lava_callback - INFO - Received callback for node 69c4831d590270ad7b6d934e INFO: 172.21.0.1:42140 - "POST /node/69c4831d590270ad7b6d934e HTTP/1.0" 202 Accepted 2026-03-26 00:52:00,987 - lava_callback - INFO - Starting async job submit for node 69c4831d590270ad7b6d934e 2026-03-26 00:52:00,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:52:01,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4831d590270ad7b6d934e HTTP/1.1" 200 None 2026-03-26 00:52:01,021 - lava_callback - DEBUG - Processing node 69c4831d590270ad7b6d934e: baseline-arm 2026-03-26 00:52:01,029 - lava_callback - WARNING - Failed to upload log 2026-03-26 00:52:01,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:52:01,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:52:01,362 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c4831d590270ad7b6d934e/lava-logs.yaml 2026-03-26 00:52:01,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:52:01,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:52:01,748 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c4831d590270ad7b6d934e/lava_callback.json.gz 2026-03-26 00:52:01,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:52:01,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4831d590270ad7b6d934e HTTP/1.1" 200 None 2026-03-26 00:52:01,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:52:01,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:52:01,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:52:01,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4831d590270ad7b6d934e HTTP/1.1" 200 None 2026-03-26 00:52:01,865 - lava_callback - INFO - Completed processing callback for node 69c4831d590270ad7b6d934e INFO: 172.21.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:52:29,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:52:29,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:53:09,879 - lava_callback - INFO - Received callback for node 69c480b8590270ad7b6d919c INFO: 172.21.0.1:51458 - "POST /node/69c480b8590270ad7b6d919c HTTP/1.0" 202 Accepted 2026-03-26 00:53:09,885 - lava_callback - INFO - Starting async job submit for node 69c480b8590270ad7b6d919c 2026-03-26 00:53:09,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:53:09,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b8590270ad7b6d919c HTTP/1.1" 200 None 2026-03-26 00:53:09,925 - lava_callback - DEBUG - Processing node 69c480b8590270ad7b6d919c: ltp-smoke 2026-03-26 00:53:10,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:53:10,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:53:10,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c480b8590270ad7b6d919c/log.txt.gz 2026-03-26 00:53:10,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:53:11,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:53:11,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c480b8590270ad7b6d919c/lava-logs.yaml 2026-03-26 00:53:11,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:53:11,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:53:11,198 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c480b8590270ad7b6d919c/lava_callback.json.gz 2026-03-26 00:53:11,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:53:11,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b8590270ad7b6d919c HTTP/1.1" 200 None 2026-03-26 00:53:11,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:53:11,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:53:11,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:53:11,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480b8590270ad7b6d919c HTTP/1.1" 200 None 2026-03-26 00:53:11,279 - lava_callback - INFO - Completed processing callback for node 69c480b8590270ad7b6d919c INFO: 172.21.0.1:51466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52584 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:53:29,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:53:29,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:54:19,612 - lava_callback - INFO - Received callback for node 69c480bb590270ad7b6d91ab INFO: 172.21.0.1:41068 - "POST /node/69c480bb590270ad7b6d91ab HTTP/1.0" 202 Accepted 2026-03-26 00:54:19,622 - lava_callback - INFO - Starting async job submit for node 69c480bb590270ad7b6d91ab 2026-03-26 00:54:19,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:19,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480bb590270ad7b6d91ab HTTP/1.1" 200 None 2026-03-26 00:54:19,687 - lava_callback - DEBUG - Processing node 69c480bb590270ad7b6d91ab: ltp-syscalls-ipc 2026-03-26 00:54:20,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:20,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:20,289 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c480bb590270ad7b6d91ab/log.txt.gz 2026-03-26 00:54:20,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:20,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:20,784 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c480bb590270ad7b6d91ab/lava-logs.yaml 2026-03-26 00:54:20,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:21,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:21,029 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c480bb590270ad7b6d91ab/lava_callback.json.gz 2026-03-26 00:54:21,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:21,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480bb590270ad7b6d91ab HTTP/1.1" 200 None 2026-03-26 00:54:21,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:21,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:54:21,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:21,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480bb590270ad7b6d91ab HTTP/1.1" 200 None 2026-03-26 00:54:21,154 - lava_callback - INFO - Completed processing callback for node 69c480bb590270ad7b6d91ab INFO: 172.21.0.1:49470 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:54:29,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:29,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 00:54:36,508 - lava_callback - INFO - Received callback for node 69c48067590270ad7b6d9080 INFO: 172.21.0.1:54534 - "POST /node/69c48067590270ad7b6d9080 HTTP/1.0" 202 Accepted 2026-03-26 00:54:36,516 - lava_callback - INFO - Starting async job submit for node 69c48067590270ad7b6d9080 INFO: 172.21.0.1:54544 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:54:36,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:36,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48067590270ad7b6d9080 HTTP/1.1" 200 None 2026-03-26 00:54:36,570 - lava_callback - DEBUG - Processing node 69c48067590270ad7b6d9080: baseline-x86 2026-03-26 00:54:36,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:37,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:37,136 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c48067590270ad7b6d9080/log.txt.gz 2026-03-26 00:54:37,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:37,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:37,524 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c48067590270ad7b6d9080/lava-logs.yaml 2026-03-26 00:54:37,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:37,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:37,691 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c48067590270ad7b6d9080/lava_callback.json.gz 2026-03-26 00:54:37,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:37,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48067590270ad7b6d9080 HTTP/1.1" 200 None 2026-03-26 00:54:37,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:37,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b5e590270ad7b6d69bd HTTP/1.1" 200 None 2026-03-26 00:54:37,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:37,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48067590270ad7b6d9080 HTTP/1.1" 200 None 2026-03-26 00:54:37,808 - lava_callback - INFO - Completed processing callback for node 69c48067590270ad7b6d9080 INFO: 172.21.0.1:48384 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:54:49,855 - lava_callback - INFO - Received callback for node 69c480bd590270ad7b6d91b1 INFO: 172.21.0.1:40740 - "POST /node/69c480bd590270ad7b6d91b1 HTTP/1.0" 202 Accepted 2026-03-26 00:54:49,863 - lava_callback - INFO - Starting async job submit for node 69c480bd590270ad7b6d91b1 2026-03-26 00:54:49,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:49,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480bd590270ad7b6d91b1 HTTP/1.1" 200 None 2026-03-26 00:54:49,913 - lava_callback - DEBUG - Processing node 69c480bd590270ad7b6d91b1: ltp-timers 2026-03-26 00:54:50,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:50,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:50,479 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c480bd590270ad7b6d91b1/log.txt.gz 2026-03-26 00:54:50,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:50,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:50,962 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c480bd590270ad7b6d91b1/lava-logs.yaml 2026-03-26 00:54:50,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:54:51,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:54:51,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c480bd590270ad7b6d91b1/lava_callback.json.gz 2026-03-26 00:54:51,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:51,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480bd590270ad7b6d91b1 HTTP/1.1" 200 None 2026-03-26 00:54:51,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:51,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:54:51,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:51,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480bd590270ad7b6d91b1 HTTP/1.1" 200 None 2026-03-26 00:54:51,211 - lava_callback - INFO - Completed processing callback for node 69c480bd590270ad7b6d91b1 INFO: 172.21.0.1:40756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40764 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:54:58,961 - lava_callback - INFO - Received callback for node 69c1e0cd0efaf21d93105ec1 INFO: 172.21.0.1:40776 - "POST /node/69c1e0cd0efaf21d93105ec1 HTTP/1.0" 202 Accepted 2026-03-26 00:54:58,975 - lava_callback - INFO - Starting async job submit for node 69c1e0cd0efaf21d93105ec1 2026-03-26 00:54:58,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:59,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e0cd0efaf21d93105ec1 HTTP/1.1" 200 None 2026-03-26 00:54:59,020 - lava_callback - DEBUG - Processing node 69c1e0cd0efaf21d93105ec1: baseline-arm-clabbe 2026-03-26 00:54:59,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e0cd0efaf21d93105ec1 2026-03-26 00:54:59,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:59,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e0cd0efaf21d93105ec1 HTTP/1.1" 200 None 2026-03-26 00:54:59,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:59,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 00:54:59,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:59,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e0cd0efaf21d93105ec1 HTTP/1.1" 200 None 2026-03-26 00:54:59,092 - lava_callback - ERROR - Error processing callback for node 69c1e0cd0efaf21d93105ec1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 00:54:59,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:54:59,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60766 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:55:38,903 - lava_callback - INFO - Received callback for node 69c1e0f50efaf21d93105eda INFO: 172.21.0.1:60774 - "POST /node/69c1e0f50efaf21d93105eda HTTP/1.0" 202 Accepted 2026-03-26 00:55:38,909 - lava_callback - INFO - Starting async job submit for node 69c1e0f50efaf21d93105eda 2026-03-26 00:55:38,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:38,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e0f50efaf21d93105eda HTTP/1.1" 200 None 2026-03-26 00:55:38,925 - lava_callback - DEBUG - Processing node 69c1e0f50efaf21d93105eda: baseline-arm-clabbe 2026-03-26 00:55:38,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e0f50efaf21d93105eda 2026-03-26 00:55:38,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:38,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e0f50efaf21d93105eda HTTP/1.1" 200 None 2026-03-26 00:55:38,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:38,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 00:55:38,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:39,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e0f50efaf21d93105eda HTTP/1.1" 200 None 2026-03-26 00:55:39,015 - lava_callback - ERROR - Error processing callback for node 69c1e0f50efaf21d93105eda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58678 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:55:49,626 - lava_callback - INFO - Received callback for node 69c480b7590270ad7b6d9189 INFO: 172.21.0.1:58680 - "POST /node/69c480b7590270ad7b6d9189 HTTP/1.0" 202 Accepted 2026-03-26 00:55:49,633 - lava_callback - INFO - Starting async job submit for node 69c480b7590270ad7b6d9189 2026-03-26 00:55:49,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:49,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b7590270ad7b6d9189 HTTP/1.1" 200 None 2026-03-26 00:55:49,703 - lava_callback - DEBUG - Processing node 69c480b7590270ad7b6d9189: ltp-sched 2026-03-26 00:55:50,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:55:50,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:55:50,507 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c480b7590270ad7b6d9189/log.txt.gz 2026-03-26 00:55:50,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:55:51,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:55:51,022 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c480b7590270ad7b6d9189/lava-logs.yaml 2026-03-26 00:55:51,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:55:51,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:55:51,256 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c480b7590270ad7b6d9189/lava_callback.json.gz 2026-03-26 00:55:51,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:51,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b7590270ad7b6d9189 HTTP/1.1" 200 None 2026-03-26 00:55:51,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:51,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:55:51,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:51,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480b7590270ad7b6d9189 HTTP/1.1" 200 None 2026-03-26 00:55:51,389 - lava_callback - INFO - Completed processing callback for node 69c480b7590270ad7b6d9189 INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:55:59,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:55:59,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43474 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:56:18,951 - lava_callback - INFO - Received callback for node 69c1e11d0efaf21d93105ee6 INFO: 172.21.0.1:43486 - "POST /node/69c1e11d0efaf21d93105ee6 HTTP/1.0" 202 Accepted 2026-03-26 00:56:18,957 - lava_callback - INFO - Starting async job submit for node 69c1e11d0efaf21d93105ee6 2026-03-26 00:56:18,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:18,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e11d0efaf21d93105ee6 HTTP/1.1" 200 None 2026-03-26 00:56:18,973 - lava_callback - DEBUG - Processing node 69c1e11d0efaf21d93105ee6: baseline-arm-clabbe 2026-03-26 00:56:18,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e11d0efaf21d93105ee6 2026-03-26 00:56:18,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:18,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e11d0efaf21d93105ee6 HTTP/1.1" 200 None 2026-03-26 00:56:18,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:19,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 00:56:19,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:19,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e11d0efaf21d93105ee6 HTTP/1.1" 200 None 2026-03-26 00:56:19,030 - lava_callback - ERROR - Error processing callback for node 69c1e11d0efaf21d93105ee6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42170 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:56:58,983 - lava_callback - INFO - Received callback for node 69c1e1460efaf21d93105efc INFO: 172.21.0.1:42172 - "POST /node/69c1e1460efaf21d93105efc HTTP/1.0" 202 Accepted 2026-03-26 00:56:58,988 - lava_callback - INFO - Starting async job submit for node 69c1e1460efaf21d93105efc 2026-03-26 00:56:58,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:59,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e1460efaf21d93105efc HTTP/1.1" 200 None 2026-03-26 00:56:59,005 - lava_callback - DEBUG - Processing node 69c1e1460efaf21d93105efc: baseline-arm-clabbe 2026-03-26 00:56:59,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e1460efaf21d93105efc 2026-03-26 00:56:59,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:59,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e1460efaf21d93105efc HTTP/1.1" 200 None 2026-03-26 00:56:59,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:59,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 00:56:59,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:56:59,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e1460efaf21d93105efc HTTP/1.1" 200 None 2026-03-26 00:56:59,080 - lava_callback - ERROR - Error processing callback for node 69c1e1460efaf21d93105efc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49768 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:57:09,974 - lava_callback - INFO - Received callback for node 69c480be590270ad7b6d91b4 INFO: 172.21.0.1:39406 - "POST /node/69c480be590270ad7b6d91b4 HTTP/1.0" 202 Accepted 2026-03-26 00:57:09,980 - lava_callback - INFO - Starting async job submit for node 69c480be590270ad7b6d91b4 2026-03-26 00:57:10,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:10,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480be590270ad7b6d91b4 HTTP/1.1" 200 None 2026-03-26 00:57:10,021 - lava_callback - DEBUG - Processing node 69c480be590270ad7b6d91b4: ltp-watchqueue 2026-03-26 00:57:10,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:57:10,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:57:10,545 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c480be590270ad7b6d91b4/log.txt.gz 2026-03-26 00:57:10,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:57:10,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:57:10,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c480be590270ad7b6d91b4/lava-logs.yaml 2026-03-26 00:57:11,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:57:11,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:57:11,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c480be590270ad7b6d91b4/lava_callback.json.gz 2026-03-26 00:57:11,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:11,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480be590270ad7b6d91b4 HTTP/1.1" 200 None 2026-03-26 00:57:11,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:11,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:57:11,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:11,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480be590270ad7b6d91b4 HTTP/1.1" 200 None 2026-03-26 00:57:11,284 - lava_callback - INFO - Completed processing callback for node 69c480be590270ad7b6d91b4 INFO: 172.21.0.1:39412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:57:29,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:29,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:57:39,024 - lava_callback - INFO - Received callback for node 69c1e16d0efaf21d93105f17 INFO: 172.21.0.1:37918 - "POST /node/69c1e16d0efaf21d93105f17 HTTP/1.0" 202 Accepted 2026-03-26 00:57:39,029 - lava_callback - INFO - Starting async job submit for node 69c1e16d0efaf21d93105f17 2026-03-26 00:57:39,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:39,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e16d0efaf21d93105f17 HTTP/1.1" 200 None 2026-03-26 00:57:39,060 - lava_callback - DEBUG - Processing node 69c1e16d0efaf21d93105f17: baseline-arm-clabbe 2026-03-26 00:57:39,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e16d0efaf21d93105f17 2026-03-26 00:57:39,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:39,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e16d0efaf21d93105f17 HTTP/1.1" 200 None 2026-03-26 00:57:39,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:39,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 00:57:39,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:57:39,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e16d0efaf21d93105f17 HTTP/1.1" 200 None 2026-03-26 00:57:39,127 - lava_callback - ERROR - Error processing callback for node 69c1e16d0efaf21d93105f17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50506 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:58:27,726 - lava_callback - INFO - Received callback for node 69c48070590270ad7b6d9094 INFO: 172.21.0.1:50520 - "POST /node/69c48070590270ad7b6d9094 HTTP/1.0" 202 Accepted 2026-03-26 00:58:27,733 - lava_callback - INFO - Starting async job submit for node 69c48070590270ad7b6d9094 2026-03-26 00:58:27,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:58:27,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48070590270ad7b6d9094 HTTP/1.1" 200 None 2026-03-26 00:58:27,844 - lava_callback - DEBUG - Processing node 69c48070590270ad7b6d9094: baseline-arm 2026-03-26 00:58:28,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:58:28,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:58:28,763 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c48070590270ad7b6d9094/log.txt.gz 2026-03-26 00:58:29,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:58:29,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:58:29,550 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c48070590270ad7b6d9094/lava-logs.yaml 2026-03-26 00:58:29,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:58:29,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:58:29,731 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c48070590270ad7b6d9094/lava_callback.json.gz DEBUG: Setup failed for 69c48070590270ad7b6d9094. Transitting job node result: incomplete -> fail 2026-03-26 00:58:29,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:58:29,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48070590270ad7b6d9094 HTTP/1.1" 200 None 2026-03-26 00:58:29,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:58:29,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:58:29,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:58:29,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48070590270ad7b6d9094 HTTP/1.1" 200 None 2026-03-26 00:58:30,135 - lava_callback - INFO - Completed processing callback for node 69c48070590270ad7b6d9094 INFO: 172.21.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42352 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:58:59,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:58:59,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44404 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:59:28,132 - lava_callback - INFO - Received callback for node 69c484d0590270ad7b6d9401 INFO: 172.21.0.1:44414 - "POST /node/69c484d0590270ad7b6d9401 HTTP/1.0" 202 Accepted 2026-03-26 00:59:28,138 - lava_callback - INFO - Starting async job submit for node 69c484d0590270ad7b6d9401 2026-03-26 00:59:28,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:28,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c484d0590270ad7b6d9401 HTTP/1.1" 200 None 2026-03-26 00:59:28,162 - lava_callback - DEBUG - Processing node 69c484d0590270ad7b6d9401: baseline-arm 2026-03-26 00:59:28,168 - lava_callback - WARNING - Failed to upload log 2026-03-26 00:59:28,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:59:29,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:59:29,303 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c484d0590270ad7b6d9401/lava-logs.yaml 2026-03-26 00:59:29,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 00:59:29,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 00:59:29,576 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c484d0590270ad7b6d9401/lava_callback.json.gz 2026-03-26 00:59:29,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:29,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c484d0590270ad7b6d9401 HTTP/1.1" 200 None 2026-03-26 00:59:29,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:29,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 00:59:29,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:29,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c484d0590270ad7b6d9401 HTTP/1.1" 200 None 2026-03-26 00:59:29,648 - lava_callback - INFO - Completed processing callback for node 69c484d0590270ad7b6d9401 2026-03-26 00:59:30,513 - lava_callback - INFO - Received callback for node 69c1e1e50efaf21d93105f3e INFO: 172.21.0.1:44906 - "POST /node/69c1e1e50efaf21d93105f3e HTTP/1.0" 202 Accepted 2026-03-26 00:59:30,521 - lava_callback - INFO - Starting async job submit for node 69c1e1e50efaf21d93105f3e 2026-03-26 00:59:30,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e1e50efaf21d93105f3e HTTP/1.1" 200 None 2026-03-26 00:59:30,547 - lava_callback - DEBUG - Processing node 69c1e1e50efaf21d93105f3e: baseline-arm-baylibre 2026-03-26 00:59:30,547 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e1e50efaf21d93105f3e 2026-03-26 00:59:30,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:30,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e1e50efaf21d93105f3e HTTP/1.1" 200 None 2026-03-26 00:59:30,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:30,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 00:59:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:30,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e1e50efaf21d93105f3e HTTP/1.1" 200 None 2026-03-26 00:59:30,665 - lava_callback - ERROR - Error processing callback for node 69c1e1e50efaf21d93105f3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39858 - "GET / HTTP/1.0" 200 OK 2026-03-26 00:59:59,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 00:59:59,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45724 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:00:10,464 - lava_callback - INFO - Received callback for node 69c1e20d0efaf21d93105f4d INFO: 172.21.0.1:53350 - "POST /node/69c1e20d0efaf21d93105f4d HTTP/1.0" 202 Accepted 2026-03-26 01:00:10,473 - lava_callback - INFO - Starting async job submit for node 69c1e20d0efaf21d93105f4d 2026-03-26 01:00:10,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:10,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e20d0efaf21d93105f4d HTTP/1.1" 200 None 2026-03-26 01:00:10,512 - lava_callback - DEBUG - Processing node 69c1e20d0efaf21d93105f4d: baseline-arm-baylibre 2026-03-26 01:00:10,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e20d0efaf21d93105f4d 2026-03-26 01:00:10,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:10,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e20d0efaf21d93105f4d HTTP/1.1" 200 None 2026-03-26 01:00:10,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:10,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:00:10,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:10,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e20d0efaf21d93105f4d HTTP/1.1" 200 None 2026-03-26 01:00:10,626 - lava_callback - ERROR - Error processing callback for node 69c1e20d0efaf21d93105f4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37962 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:00:28,773 - lava_callback - INFO - Received callback for node 69c48095590270ad7b6d9123 INFO: 172.21.0.1:37978 - "POST /node/69c48095590270ad7b6d9123 HTTP/1.0" 202 Accepted 2026-03-26 01:00:28,824 - lava_callback - INFO - Starting async job submit for node 69c48095590270ad7b6d9123 2026-03-26 01:00:28,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:28,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48095590270ad7b6d9123 HTTP/1.1" 200 None 2026-03-26 01:00:28,914 - lava_callback - DEBUG - Processing node 69c48095590270ad7b6d9123: baseline-arm-pengutronix 2026-03-26 01:00:29,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:00:29,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:00:29,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48095590270ad7b6d9123/log.txt.gz 2026-03-26 01:00:30,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:00:30,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:00:30,442 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48095590270ad7b6d9123/lava-logs.yaml 2026-03-26 01:00:30,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:00:31,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:00:31,427 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48095590270ad7b6d9123/lava_callback.json.gz DEBUG: Setup failed for 69c48095590270ad7b6d9123. Transitting job node result: incomplete -> fail 2026-03-26 01:00:31,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:31,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48095590270ad7b6d9123 HTTP/1.1" 200 None 2026-03-26 01:00:31,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:31,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 01:00:31,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:31,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48095590270ad7b6d9123 HTTP/1.1" 200 None 2026-03-26 01:00:31,704 - lava_callback - INFO - Completed processing callback for node 69c48095590270ad7b6d9123 INFO: 172.21.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35146 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:00:50,500 - lava_callback - INFO - Received callback for node 69c1e2360efaf21d93105f5e INFO: 172.21.0.1:53636 - "POST /node/69c1e2360efaf21d93105f5e HTTP/1.0" 202 Accepted 2026-03-26 01:00:50,508 - lava_callback - INFO - Starting async job submit for node 69c1e2360efaf21d93105f5e 2026-03-26 01:00:50,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:50,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2360efaf21d93105f5e HTTP/1.1" 200 None 2026-03-26 01:00:50,535 - lava_callback - DEBUG - Processing node 69c1e2360efaf21d93105f5e: baseline-arm-baylibre 2026-03-26 01:00:50,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e2360efaf21d93105f5e 2026-03-26 01:00:50,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:50,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2360efaf21d93105f5e HTTP/1.1" 200 None 2026-03-26 01:00:50,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:50,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:00:50,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:50,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e2360efaf21d93105f5e HTTP/1.1" 200 None 2026-03-26 01:00:50,631 - lava_callback - ERROR - Error processing callback for node 69c1e2360efaf21d93105f5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53658 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:00:59,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:00:59,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:02:10,707 - lava_callback - INFO - Received callback for node 69c1e2860efaf21d93105f6f INFO: 172.21.0.1:41252 - "POST /node/69c1e2860efaf21d93105f6f HTTP/1.0" 202 Accepted 2026-03-26 01:02:10,715 - lava_callback - INFO - Starting async job submit for node 69c1e2860efaf21d93105f6f 2026-03-26 01:02:10,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:10,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2860efaf21d93105f6f HTTP/1.1" 200 None 2026-03-26 01:02:10,761 - lava_callback - DEBUG - Processing node 69c1e2860efaf21d93105f6f: baseline-arm-baylibre 2026-03-26 01:02:10,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e2860efaf21d93105f6f 2026-03-26 01:02:10,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:10,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2860efaf21d93105f6f HTTP/1.1" 200 None 2026-03-26 01:02:10,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:10,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:02:10,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:10,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e2860efaf21d93105f6f HTTP/1.1" 200 None 2026-03-26 01:02:10,857 - lava_callback - ERROR - Error processing callback for node 69c1e2860efaf21d93105f6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:02:50,603 - lava_callback - INFO - Received callback for node 69c1e2af0efaf21d93105f7e INFO: 172.21.0.1:42952 - "POST /node/69c1e2af0efaf21d93105f7e HTTP/1.0" 202 Accepted 2026-03-26 01:02:50,626 - lava_callback - INFO - Starting async job submit for node 69c1e2af0efaf21d93105f7e 2026-03-26 01:02:50,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:50,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2af0efaf21d93105f7e HTTP/1.1" 200 None 2026-03-26 01:02:50,651 - lava_callback - DEBUG - Processing node 69c1e2af0efaf21d93105f7e: baseline-arm-baylibre 2026-03-26 01:02:50,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e2af0efaf21d93105f7e 2026-03-26 01:02:50,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:50,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2af0efaf21d93105f7e HTTP/1.1" 200 None 2026-03-26 01:02:50,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:50,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:02:50,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:02:50,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e2af0efaf21d93105f7e HTTP/1.1" 200 None 2026-03-26 01:02:50,805 - lava_callback - ERROR - Error processing callback for node 69c1e2af0efaf21d93105f7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39484 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:03:30,641 - lava_callback - INFO - Received callback for node 69c1e2d60efaf21d93105f8e INFO: 172.21.0.1:35702 - "POST /node/69c1e2d60efaf21d93105f8e HTTP/1.0" 202 Accepted 2026-03-26 01:03:30,648 - lava_callback - INFO - Starting async job submit for node 69c1e2d60efaf21d93105f8e 2026-03-26 01:03:30,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:03:30,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2d60efaf21d93105f8e HTTP/1.1" 200 None 2026-03-26 01:03:30,667 - lava_callback - DEBUG - Processing node 69c1e2d60efaf21d93105f8e: baseline-arm-baylibre 2026-03-26 01:03:30,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e2d60efaf21d93105f8e 2026-03-26 01:03:30,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:03:30,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e2d60efaf21d93105f8e HTTP/1.1" 200 None 2026-03-26 01:03:30,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:03:30,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:03:30,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:03:30,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e2d60efaf21d93105f8e HTTP/1.1" 200 None 2026-03-26 01:03:30,747 - lava_callback - ERROR - Error processing callback for node 69c1e2d60efaf21d93105f8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41924 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:04:39,325 - lava_callback - INFO - Received callback for node 69c1e3180efaf21d93105f9c INFO: 172.21.0.1:41938 - "POST /node/69c1e3180efaf21d93105f9c HTTP/1.0" 202 Accepted 2026-03-26 01:04:39,332 - lava_callback - INFO - Starting async job submit for node 69c1e3180efaf21d93105f9c 2026-03-26 01:04:39,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:04:39,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3180efaf21d93105f9c HTTP/1.1" 200 None 2026-03-26 01:04:39,352 - lava_callback - DEBUG - Processing node 69c1e3180efaf21d93105f9c: baseline-arm-clabbe 2026-03-26 01:04:39,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3180efaf21d93105f9c 2026-03-26 01:04:39,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:04:39,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3180efaf21d93105f9c HTTP/1.1" 200 None 2026-03-26 01:04:39,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:04:39,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:04:39,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:04:39,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3180efaf21d93105f9c HTTP/1.1" 200 None 2026-03-26 01:04:39,416 - lava_callback - ERROR - Error processing callback for node 69c1e3180efaf21d93105f9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48918 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:05:19,332 - lava_callback - INFO - Received callback for node 69c1e3410efaf21d93105fac INFO: 172.21.0.1:48922 - "POST /node/69c1e3410efaf21d93105fac HTTP/1.0" 202 Accepted 2026-03-26 01:05:19,340 - lava_callback - INFO - Starting async job submit for node 69c1e3410efaf21d93105fac 2026-03-26 01:05:19,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:19,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3410efaf21d93105fac HTTP/1.1" 200 None 2026-03-26 01:05:19,358 - lava_callback - DEBUG - Processing node 69c1e3410efaf21d93105fac: baseline-arm-clabbe 2026-03-26 01:05:19,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3410efaf21d93105fac 2026-03-26 01:05:19,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:19,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3410efaf21d93105fac HTTP/1.1" 200 None 2026-03-26 01:05:19,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:19,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:05:19,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:19,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3410efaf21d93105fac HTTP/1.1" 200 None 2026-03-26 01:05:19,420 - lava_callback - ERROR - Error processing callback for node 69c1e3410efaf21d93105fac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:05:59,370 - lava_callback - INFO - Received callback for node 69c1e36a0efaf21d93105fc0 INFO: 172.21.0.1:34270 - "POST /node/69c1e36a0efaf21d93105fc0 HTTP/1.0" 202 Accepted 2026-03-26 01:05:59,375 - lava_callback - INFO - Starting async job submit for node 69c1e36a0efaf21d93105fc0 2026-03-26 01:05:59,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:59,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e36a0efaf21d93105fc0 HTTP/1.1" 200 None 2026-03-26 01:05:59,392 - lava_callback - DEBUG - Processing node 69c1e36a0efaf21d93105fc0: baseline-arm-clabbe 2026-03-26 01:05:59,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e36a0efaf21d93105fc0 2026-03-26 01:05:59,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:59,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e36a0efaf21d93105fc0 HTTP/1.1" 200 None 2026-03-26 01:05:59,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:59,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:05:59,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:05:59,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e36a0efaf21d93105fc0 HTTP/1.1" 200 None 2026-03-26 01:05:59,452 - lava_callback - ERROR - Error processing callback for node 69c1e36a0efaf21d93105fc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52344 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:06:39,395 - lava_callback - INFO - Received callback for node 69c1e3940efaf21d93106020 INFO: 172.21.0.1:52356 - "POST /node/69c1e3940efaf21d93106020 HTTP/1.0" 202 Accepted 2026-03-26 01:06:39,401 - lava_callback - INFO - Starting async job submit for node 69c1e3940efaf21d93106020 2026-03-26 01:06:39,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:39,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3940efaf21d93106020 HTTP/1.1" 200 None 2026-03-26 01:06:39,417 - lava_callback - DEBUG - Processing node 69c1e3940efaf21d93106020: baseline-arm-clabbe 2026-03-26 01:06:39,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3940efaf21d93106020 2026-03-26 01:06:39,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:39,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3940efaf21d93106020 HTTP/1.1" 200 None 2026-03-26 01:06:39,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:39,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:06:39,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:39,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3940efaf21d93106020 HTTP/1.1" 200 None 2026-03-26 01:06:39,488 - lava_callback - ERROR - Error processing callback for node 69c1e3940efaf21d93106020: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41776 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:06:50,858 - lava_callback - INFO - Received callback for node 69c1e3a90efaf21d9310606c INFO: 172.21.0.1:43324 - "POST /node/69c1e3a90efaf21d9310606c HTTP/1.0" 202 Accepted 2026-03-26 01:06:50,863 - lava_callback - INFO - Starting async job submit for node 69c1e3a90efaf21d9310606c 2026-03-26 01:06:50,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:50,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a90efaf21d9310606c HTTP/1.1" 200 None 2026-03-26 01:06:50,885 - lava_callback - DEBUG - Processing node 69c1e3a90efaf21d9310606c: baseline-arm64-baylibre 2026-03-26 01:06:50,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3a90efaf21d9310606c 2026-03-26 01:06:50,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:50,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3a90efaf21d9310606c HTTP/1.1" 200 None 2026-03-26 01:06:50,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:50,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:06:50,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:06:50,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3a90efaf21d9310606c HTTP/1.1" 200 None 2026-03-26 01:06:50,937 - lava_callback - ERROR - Error processing callback for node 69c1e3a90efaf21d9310606c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:07:10,807 - lava_callback - INFO - Received callback for node 69c1e3aa0efaf21d93106071 INFO: 172.21.0.1:41202 - "POST /node/69c1e3aa0efaf21d93106071 HTTP/1.0" 202 Accepted 2026-03-26 01:07:10,816 - lava_callback - INFO - Starting async job submit for node 69c1e3aa0efaf21d93106071 2026-03-26 01:07:10,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:10,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3aa0efaf21d93106071 HTTP/1.1" 200 None 2026-03-26 01:07:10,847 - lava_callback - DEBUG - Processing node 69c1e3aa0efaf21d93106071: baseline-arm64-baylibre 2026-03-26 01:07:10,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3aa0efaf21d93106071 2026-03-26 01:07:10,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:10,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3aa0efaf21d93106071 HTTP/1.1" 200 None 2026-03-26 01:07:10,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:10,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:10,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:10,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3aa0efaf21d93106071 HTTP/1.1" 200 None 2026-03-26 01:07:10,937 - lava_callback - ERROR - Error processing callback for node 69c1e3aa0efaf21d93106071: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:11,504 - lava_callback - INFO - Received callback for node 69c1e3ac0efaf21d93106074 INFO: 172.21.0.1:41214 - "POST /node/69c1e3ac0efaf21d93106074 HTTP/1.0" 202 Accepted 2026-03-26 01:07:11,513 - lava_callback - INFO - Starting async job submit for node 69c1e3ac0efaf21d93106074 2026-03-26 01:07:11,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:11,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ac0efaf21d93106074 HTTP/1.1" 200 None 2026-03-26 01:07:11,543 - lava_callback - DEBUG - Processing node 69c1e3ac0efaf21d93106074: baseline-arm64-baylibre 2026-03-26 01:07:11,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3ac0efaf21d93106074 2026-03-26 01:07:11,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:11,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ac0efaf21d93106074 HTTP/1.1" 200 None 2026-03-26 01:07:11,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:11,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:11,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:11,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3ac0efaf21d93106074 HTTP/1.1" 200 None 2026-03-26 01:07:11,626 - lava_callback - ERROR - Error processing callback for node 69c1e3ac0efaf21d93106074: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:12,194 - lava_callback - INFO - Received callback for node 69c1e3ae0efaf21d9310607b INFO: 172.21.0.1:41218 - "POST /node/69c1e3ae0efaf21d9310607b HTTP/1.0" 202 Accepted 2026-03-26 01:07:12,199 - lava_callback - INFO - Starting async job submit for node 69c1e3ae0efaf21d9310607b 2026-03-26 01:07:12,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:12,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ae0efaf21d9310607b HTTP/1.1" 200 None 2026-03-26 01:07:12,216 - lava_callback - DEBUG - Processing node 69c1e3ae0efaf21d9310607b: baseline-arm64-baylibre 2026-03-26 01:07:12,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3ae0efaf21d9310607b 2026-03-26 01:07:12,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:12,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3ae0efaf21d9310607b HTTP/1.1" 200 None 2026-03-26 01:07:12,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:12,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:12,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:12,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3ae0efaf21d9310607b HTTP/1.1" 200 None 2026-03-26 01:07:12,289 - lava_callback - ERROR - Error processing callback for node 69c1e3ae0efaf21d9310607b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:12,867 - lava_callback - INFO - Received callback for node 69c1e3b00efaf21d9310607e INFO: 172.21.0.1:41228 - "POST /node/69c1e3b00efaf21d9310607e HTTP/1.0" 202 Accepted 2026-03-26 01:07:12,876 - lava_callback - INFO - Starting async job submit for node 69c1e3b00efaf21d9310607e 2026-03-26 01:07:12,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:12,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b00efaf21d9310607e HTTP/1.1" 200 None 2026-03-26 01:07:12,902 - lava_callback - DEBUG - Processing node 69c1e3b00efaf21d9310607e: baseline-arm64-baylibre 2026-03-26 01:07:12,902 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3b00efaf21d9310607e 2026-03-26 01:07:12,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:12,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b00efaf21d9310607e HTTP/1.1" 200 None 2026-03-26 01:07:12,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:12,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:12,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:13,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3b00efaf21d9310607e HTTP/1.1" 200 None 2026-03-26 01:07:13,004 - lava_callback - ERROR - Error processing callback for node 69c1e3b00efaf21d9310607e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:13,561 - lava_callback - INFO - Received callback for node 69c1e3b10efaf21d93106081 INFO: 172.21.0.1:41232 - "POST /node/69c1e3b10efaf21d93106081 HTTP/1.0" 202 Accepted 2026-03-26 01:07:13,567 - lava_callback - INFO - Starting async job submit for node 69c1e3b10efaf21d93106081 2026-03-26 01:07:13,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:13,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b10efaf21d93106081 HTTP/1.1" 200 None 2026-03-26 01:07:13,586 - lava_callback - DEBUG - Processing node 69c1e3b10efaf21d93106081: baseline-arm64-baylibre 2026-03-26 01:07:13,587 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3b10efaf21d93106081 2026-03-26 01:07:13,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:13,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b10efaf21d93106081 HTTP/1.1" 200 None 2026-03-26 01:07:13,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:13,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:13,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:13,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3b10efaf21d93106081 HTTP/1.1" 200 None 2026-03-26 01:07:13,670 - lava_callback - ERROR - Error processing callback for node 69c1e3b10efaf21d93106081: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41244 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:07:19,427 - lava_callback - INFO - Received callback for node 69c1e3c10efaf21d931060b1 INFO: 172.21.0.1:41252 - "POST /node/69c1e3c10efaf21d931060b1 HTTP/1.0" 202 Accepted 2026-03-26 01:07:19,436 - lava_callback - INFO - Starting async job submit for node 69c1e3c10efaf21d931060b1 2026-03-26 01:07:19,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:19,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c10efaf21d931060b1 HTTP/1.1" 200 None 2026-03-26 01:07:19,469 - lava_callback - DEBUG - Processing node 69c1e3c10efaf21d931060b1: baseline-arm64-clabbe 2026-03-26 01:07:19,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3c10efaf21d931060b1 2026-03-26 01:07:19,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:19,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c10efaf21d931060b1 HTTP/1.1" 200 None 2026-03-26 01:07:19,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:19,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:19,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:19,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c10efaf21d931060b1 HTTP/1.1" 200 None 2026-03-26 01:07:19,571 - lava_callback - ERROR - Error processing callback for node 69c1e3c10efaf21d931060b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:20,106 - lava_callback - INFO - Received callback for node 69c1e3c20efaf21d931060ce INFO: 172.21.0.1:37754 - "POST /node/69c1e3c20efaf21d931060ce HTTP/1.0" 202 Accepted 2026-03-26 01:07:20,116 - lava_callback - INFO - Starting async job submit for node 69c1e3c20efaf21d931060ce 2026-03-26 01:07:20,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c20efaf21d931060ce HTTP/1.1" 200 None 2026-03-26 01:07:20,144 - lava_callback - DEBUG - Processing node 69c1e3c20efaf21d931060ce: baseline-arm64-clabbe 2026-03-26 01:07:20,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3c20efaf21d931060ce 2026-03-26 01:07:20,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c20efaf21d931060ce HTTP/1.1" 200 None 2026-03-26 01:07:20,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:20,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c20efaf21d931060ce HTTP/1.1" 200 None 2026-03-26 01:07:20,216 - lava_callback - ERROR - Error processing callback for node 69c1e3c20efaf21d931060ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:20,809 - lava_callback - INFO - Received callback for node 69c1e3c30efaf21d931060d1 INFO: 172.21.0.1:37770 - "POST /node/69c1e3c30efaf21d931060d1 HTTP/1.0" 202 Accepted 2026-03-26 01:07:20,815 - lava_callback - INFO - Starting async job submit for node 69c1e3c30efaf21d931060d1 2026-03-26 01:07:20,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c30efaf21d931060d1 HTTP/1.1" 200 None 2026-03-26 01:07:20,836 - lava_callback - DEBUG - Processing node 69c1e3c30efaf21d931060d1: baseline-arm64-clabbe 2026-03-26 01:07:20,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3c30efaf21d931060d1 2026-03-26 01:07:20,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c30efaf21d931060d1 HTTP/1.1" 200 None 2026-03-26 01:07:20,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:20,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:20,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c30efaf21d931060d1 HTTP/1.1" 200 None 2026-03-26 01:07:20,894 - lava_callback - ERROR - Error processing callback for node 69c1e3c30efaf21d931060d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:21,499 - lava_callback - INFO - Received callback for node 69c1e3c40efaf21d931060d4 INFO: 172.21.0.1:37786 - "POST /node/69c1e3c40efaf21d931060d4 HTTP/1.0" 202 Accepted 2026-03-26 01:07:21,506 - lava_callback - INFO - Starting async job submit for node 69c1e3c40efaf21d931060d4 2026-03-26 01:07:21,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:21,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c40efaf21d931060d4 HTTP/1.1" 200 None 2026-03-26 01:07:21,522 - lava_callback - DEBUG - Processing node 69c1e3c40efaf21d931060d4: baseline-arm64-clabbe 2026-03-26 01:07:21,522 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3c40efaf21d931060d4 2026-03-26 01:07:21,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:21,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c40efaf21d931060d4 HTTP/1.1" 200 None 2026-03-26 01:07:21,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:21,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:21,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:21,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c40efaf21d931060d4 HTTP/1.1" 200 None 2026-03-26 01:07:21,578 - lava_callback - ERROR - Error processing callback for node 69c1e3c40efaf21d931060d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:07:22,186 - lava_callback - INFO - Received callback for node 69c1e3c60efaf21d931060da INFO: 172.21.0.1:37798 - "POST /node/69c1e3c60efaf21d931060da HTTP/1.0" 202 Accepted 2026-03-26 01:07:22,192 - lava_callback - INFO - Starting async job submit for node 69c1e3c60efaf21d931060da 2026-03-26 01:07:22,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:22,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c60efaf21d931060da HTTP/1.1" 200 None 2026-03-26 01:07:22,209 - lava_callback - DEBUG - Processing node 69c1e3c60efaf21d931060da: baseline-arm64-clabbe 2026-03-26 01:07:22,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3c60efaf21d931060da 2026-03-26 01:07:22,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:22,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c60efaf21d931060da HTTP/1.1" 200 None 2026-03-26 01:07:22,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:22,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:22,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:22,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c60efaf21d931060da HTTP/1.1" 200 None 2026-03-26 01:07:22,262 - lava_callback - ERROR - Error processing callback for node 69c1e3c60efaf21d931060da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37812 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:07:29,771 - lava_callback - INFO - Received callback for node 69c480b2590270ad7b6d917a INFO: 172.21.0.1:45804 - "POST /node/69c480b2590270ad7b6d917a HTTP/1.0" 202 Accepted 2026-03-26 01:07:29,820 - lava_callback - INFO - Starting async job submit for node 69c480b2590270ad7b6d917a 2026-03-26 01:07:29,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:29,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b2590270ad7b6d917a HTTP/1.1" 200 None 2026-03-26 01:07:29,870 - lava_callback - DEBUG - Processing node 69c480b2590270ad7b6d917a: ltp-input 2026-03-26 01:07:29,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:07:30,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:07:30,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c480b2590270ad7b6d917a/log.txt.gz 2026-03-26 01:07:30,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:07:30,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:07:30,664 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c480b2590270ad7b6d917a/lava-logs.yaml 2026-03-26 01:07:30,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:07:31,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:07:31,096 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c480b2590270ad7b6d917a/lava_callback.json.gz 2026-03-26 01:07:31,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:31,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480b2590270ad7b6d917a HTTP/1.1" 200 None 2026-03-26 01:07:31,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:31,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 01:07:31,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:31,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480b2590270ad7b6d917a HTTP/1.1" 200 None 2026-03-26 01:07:31,248 - lava_callback - INFO - Completed processing callback for node 69c480b2590270ad7b6d917a INFO: 172.21.0.1:45806 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:07:39,387 - lava_callback - INFO - Received callback for node 69c1e3c60efaf21d931060dd INFO: 172.21.0.1:45812 - "POST /node/69c1e3c60efaf21d931060dd HTTP/1.0" 202 Accepted 2026-03-26 01:07:39,393 - lava_callback - INFO - Starting async job submit for node 69c1e3c60efaf21d931060dd 2026-03-26 01:07:39,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:39,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c60efaf21d931060dd HTTP/1.1" 200 None 2026-03-26 01:07:39,410 - lava_callback - DEBUG - Processing node 69c1e3c60efaf21d931060dd: baseline-arm64-clabbe 2026-03-26 01:07:39,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e3c60efaf21d931060dd 2026-03-26 01:07:39,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:39,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3c60efaf21d931060dd HTTP/1.1" 200 None 2026-03-26 01:07:39,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:39,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 01:07:39,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:39,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3c60efaf21d931060dd HTTP/1.1" 200 None 2026-03-26 01:07:39,477 - lava_callback - ERROR - Error processing callback for node 69c1e3c60efaf21d931060dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42034 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:07:59,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:07:59,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35454 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:08:59,534 - lava_callback - INFO - Received callback for node 69c1e4160efaf21d931061b8 INFO: 172.21.0.1:35460 - "POST /node/69c1e4160efaf21d931061b8 HTTP/1.0" 202 Accepted 2026-03-26 01:08:59,540 - lava_callback - INFO - Starting async job submit for node 69c1e4160efaf21d931061b8 2026-03-26 01:08:59,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:08:59,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e4160efaf21d931061b8 HTTP/1.1" 200 None 2026-03-26 01:08:59,564 - lava_callback - DEBUG - Processing node 69c1e4160efaf21d931061b8: baseline-arm-clabbe 2026-03-26 01:08:59,564 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e4160efaf21d931061b8 2026-03-26 01:08:59,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:08:59,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e4160efaf21d931061b8 HTTP/1.1" 200 None 2026-03-26 01:08:59,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:08:59,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef76d5731cbd74145fb9 HTTP/1.1" 200 None 2026-03-26 01:08:59,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:08:59,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e4160efaf21d931061b8 HTTP/1.1" 200 None 2026-03-26 01:08:59,624 - lava_callback - ERROR - Error processing callback for node 69c1e4160efaf21d931061b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42630 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:14:23,098 - lava_callback - INFO - Received callback for node 69c48846590270ad7b6d94f8 INFO: 172.21.0.1:44012 - "POST /node/69c48846590270ad7b6d94f8 HTTP/1.0" 202 Accepted 2026-03-26 01:14:23,105 - lava_callback - INFO - Starting async job submit for node 69c48846590270ad7b6d94f8 2026-03-26 01:14:23,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:23,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48846590270ad7b6d94f8 HTTP/1.1" 200 None 2026-03-26 01:14:23,139 - lava_callback - DEBUG - Processing node 69c48846590270ad7b6d94f8: baseline-arm64 2026-03-26 01:14:23,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:23,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:23,472 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48846590270ad7b6d94f8/log.txt.gz 2026-03-26 01:14:23,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:23,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:23,758 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48846590270ad7b6d94f8/lava-logs.yaml 2026-03-26 01:14:23,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:23,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:23,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48846590270ad7b6d94f8/lava_callback.json.gz 2026-03-26 01:14:23,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:23,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48846590270ad7b6d94f8 HTTP/1.1" 200 None 2026-03-26 01:14:23,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:23,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:14:23,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:23,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48846590270ad7b6d94f8 HTTP/1.1" 200 None 2026-03-26 01:14:23,994 - lava_callback - INFO - Completed processing callback for node 69c48846590270ad7b6d94f8 2026-03-26 01:14:24,554 - lava_callback - INFO - Received callback for node 69c48861590270ad7b6d9562 INFO: 172.21.0.1:44014 - "POST /node/69c48861590270ad7b6d9562 HTTP/1.0" 202 Accepted 2026-03-26 01:14:24,560 - lava_callback - INFO - Starting async job submit for node 69c48861590270ad7b6d9562 2026-03-26 01:14:24,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:24,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48861590270ad7b6d9562 HTTP/1.1" 200 None 2026-03-26 01:14:24,579 - lava_callback - DEBUG - Processing node 69c48861590270ad7b6d9562: baseline-nfs-arm64 2026-03-26 01:14:24,580 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:14:24,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:24,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:24,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48861590270ad7b6d9562/lava-logs.yaml 2026-03-26 01:14:24,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:24,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:24,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48861590270ad7b6d9562/lava_callback.json.gz 2026-03-26 01:14:24,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:24,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48861590270ad7b6d9562 HTTP/1.1" 200 None 2026-03-26 01:14:24,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:24,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:14:24,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:24,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48861590270ad7b6d9562 HTTP/1.1" 200 None 2026-03-26 01:14:24,956 - lava_callback - INFO - Completed processing callback for node 69c48861590270ad7b6d9562 INFO: 172.21.0.1:44026 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:14:29,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:29,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 01:14:33,019 - lava_callback - INFO - Received callback for node 69c48852590270ad7b6d9519 INFO: 172.21.0.1:53148 - "POST /node/69c48852590270ad7b6d9519 HTTP/1.0" 202 Accepted 2026-03-26 01:14:33,044 - lava_callback - INFO - Starting async job submit for node 69c48852590270ad7b6d9519 2026-03-26 01:14:33,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:33,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48852590270ad7b6d9519 HTTP/1.1" 200 None 2026-03-26 01:14:33,133 - lava_callback - DEBUG - Processing node 69c48852590270ad7b6d9519: baseline-arm64 2026-03-26 01:14:33,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:33,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:33,462 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48852590270ad7b6d9519/log.txt.gz 2026-03-26 01:14:33,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:33,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:33,751 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48852590270ad7b6d9519/lava-logs.yaml 2026-03-26 01:14:33,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:33,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:33,903 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48852590270ad7b6d9519/lava_callback.json.gz 2026-03-26 01:14:33,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:33,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48852590270ad7b6d9519 HTTP/1.1" 200 None 2026-03-26 01:14:33,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:33,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:14:33,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:34,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48852590270ad7b6d9519 HTTP/1.1" 200 None 2026-03-26 01:14:34,013 - lava_callback - INFO - Completed processing callback for node 69c48852590270ad7b6d9519 INFO: 172.21.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:14:51,298 - lava_callback - INFO - Received callback for node 69c48848590270ad7b6d94fe INFO: 172.21.0.1:47070 - "POST /node/69c48848590270ad7b6d94fe HTTP/1.0" 202 Accepted 2026-03-26 01:14:51,309 - lava_callback - INFO - Starting async job submit for node 69c48848590270ad7b6d94fe 2026-03-26 01:14:51,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:51,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48848590270ad7b6d94fe HTTP/1.1" 200 None 2026-03-26 01:14:51,376 - lava_callback - DEBUG - Processing node 69c48848590270ad7b6d94fe: baseline-arm64 2026-03-26 01:14:51,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:51,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:51,867 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48848590270ad7b6d94fe/log.txt.gz 2026-03-26 01:14:52,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:47082 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:14:52,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:52,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48848590270ad7b6d94fe/lava-logs.yaml 2026-03-26 01:14:52,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:52,492 - lava_callback - INFO - Received callback for node 69c4884a590270ad7b6d9504 INFO: 172.21.0.1:47090 - "POST /node/69c4884a590270ad7b6d9504 HTTP/1.0" 202 Accepted 2026-03-26 01:14:52,500 - lava_callback - INFO - Starting async job submit for node 69c4884a590270ad7b6d9504 2026-03-26 01:14:52,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:52,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884a590270ad7b6d9504 HTTP/1.1" 200 None 2026-03-26 01:14:52,551 - lava_callback - DEBUG - Processing node 69c4884a590270ad7b6d9504: baseline-arm64 2026-03-26 01:14:52,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:52,612 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48848590270ad7b6d94fe/lava_callback.json.gz 2026-03-26 01:14:52,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:52,796 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:14:52,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48848590270ad7b6d94fe HTTP/1.1" 200 None 2026-03-26 01:14:52,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:52,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:52,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:14:52,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:52,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48848590270ad7b6d94fe HTTP/1.1" 200 None 2026-03-26 01:14:52,945 - lava_callback - INFO - Completed processing callback for node 69c48848590270ad7b6d94fe 2026-03-26 01:14:53,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:53,122 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884a590270ad7b6d9504/lava-logs.yaml 2026-03-26 01:14:53,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:53,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:53,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884a590270ad7b6d9504/lava_callback.json.gz 2026-03-26 01:14:53,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:53,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884a590270ad7b6d9504 HTTP/1.1" 200 None 2026-03-26 01:14:53,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:53,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:14:53,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:53,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4884a590270ad7b6d9504 HTTP/1.1" 200 None 2026-03-26 01:14:53,449 - lava_callback - INFO - Completed processing callback for node 69c4884a590270ad7b6d9504 2026-03-26 01:14:54,439 - lava_callback - INFO - Received callback for node 69c4884b590270ad7b6d9507 INFO: 172.21.0.1:47096 - "POST /node/69c4884b590270ad7b6d9507 HTTP/1.0" 202 Accepted 2026-03-26 01:14:54,451 - lava_callback - INFO - Starting async job submit for node 69c4884b590270ad7b6d9507 2026-03-26 01:14:54,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:54,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884b590270ad7b6d9507 HTTP/1.1" 200 None 2026-03-26 01:14:54,531 - lava_callback - DEBUG - Processing node 69c4884b590270ad7b6d9507: baseline-arm64 2026-03-26 01:14:54,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:54,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:54,843 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884b590270ad7b6d9507/log.txt.gz 2026-03-26 01:14:54,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:55,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:55,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884b590270ad7b6d9507/lava-logs.yaml 2026-03-26 01:14:55,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:55,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:55,215 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884b590270ad7b6d9507/lava_callback.json.gz 2026-03-26 01:14:55,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:55,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884b590270ad7b6d9507 HTTP/1.1" 200 None 2026-03-26 01:14:55,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:55,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:14:55,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:55,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4884b590270ad7b6d9507 HTTP/1.1" 200 None 2026-03-26 01:14:55,386 - lava_callback - INFO - Completed processing callback for node 69c4884b590270ad7b6d9507 INFO: 172.21.0.1:47106 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:14:58,461 - lava_callback - INFO - Received callback for node 69c48856590270ad7b6d9526 INFO: 172.21.0.1:47118 - "POST /node/69c48856590270ad7b6d9526 HTTP/1.0" 202 Accepted 2026-03-26 01:14:58,468 - lava_callback - INFO - Starting async job submit for node 69c48856590270ad7b6d9526 2026-03-26 01:14:58,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:58,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48856590270ad7b6d9526 HTTP/1.1" 200 None 2026-03-26 01:14:58,504 - lava_callback - DEBUG - Processing node 69c48856590270ad7b6d9526: baseline-arm64 2026-03-26 01:14:58,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:58,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:58,886 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48856590270ad7b6d9526/log.txt.gz 2026-03-26 01:14:59,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:59,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:59,226 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48856590270ad7b6d9526/lava-logs.yaml 2026-03-26 01:14:59,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:14:59,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:14:59,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48856590270ad7b6d9526/lava_callback.json.gz 2026-03-26 01:14:59,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:59,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48856590270ad7b6d9526 HTTP/1.1" 200 None 2026-03-26 01:14:59,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:59,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:14:59,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:59,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48856590270ad7b6d9526 HTTP/1.1" 200 None 2026-03-26 01:14:59,474 - lava_callback - INFO - Completed processing callback for node 69c48856590270ad7b6d9526 2026-03-26 01:14:59,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:14:59,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 01:14:59,930 - lava_callback - INFO - Received callback for node 69c48850590270ad7b6d9513 INFO: 172.21.0.1:36564 - "POST /node/69c48850590270ad7b6d9513 HTTP/1.0" 202 Accepted 2026-03-26 01:14:59,937 - lava_callback - INFO - Starting async job submit for node 69c48850590270ad7b6d9513 2026-03-26 01:14:59,958 - lava_callback - INFO - Received callback for node 69c48461590270ad7b6d93d6 2026-03-26 01:14:59,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:36570 - "POST /node/69c48461590270ad7b6d93d6 HTTP/1.0" 202 Accepted 2026-03-26 01:14:59,966 - lava_callback - INFO - Starting async job submit for node 69c48461590270ad7b6d93d6 2026-03-26 01:15:00,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:00,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48850590270ad7b6d9513 HTTP/1.1" 200 None 2026-03-26 01:15:00,034 - lava_callback - DEBUG - Processing node 69c48850590270ad7b6d9513: baseline-arm64 2026-03-26 01:15:00,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48461590270ad7b6d93d6 HTTP/1.1" 200 None 2026-03-26 01:15:00,093 - lava_callback - DEBUG - Processing node 69c48461590270ad7b6d93d6: ltp-smoke 2026-03-26 01:15:00,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:00,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:01,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:01,037 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48850590270ad7b6d9513/log.txt.gz 2026-03-26 01:15:01,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:01,140 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c48461590270ad7b6d93d6/log.txt.gz 2026-03-26 01:15:01,206 - lava_callback - INFO - Received callback for node 69c48857590270ad7b6d954a 2026-03-26 01:15:01,264 - lava_callback - INFO - Starting async job submit for node 69c48857590270ad7b6d954a INFO: 172.21.0.1:36576 - "POST /node/69c48857590270ad7b6d954a HTTP/1.0" 202 Accepted 2026-03-26 01:15:01,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:01,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:01,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48857590270ad7b6d954a HTTP/1.1" 200 None 2026-03-26 01:15:01,521 - lava_callback - DEBUG - Processing node 69c48857590270ad7b6d954a: baseline-arm64 2026-03-26 01:15:01,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:01,944 - lava_callback - INFO - Received callback for node 69c4884f590270ad7b6d9510 2026-03-26 01:15:01,954 - lava_callback - INFO - Starting async job submit for node 69c4884f590270ad7b6d9510 INFO: 172.21.0.1:36582 - "POST /node/69c4884f590270ad7b6d9510 HTTP/1.0" 202 Accepted 2026-03-26 01:15:01,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:01,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:01,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:01,993 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48850590270ad7b6d9513/lava-logs.yaml 2026-03-26 01:15:02,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884f590270ad7b6d9510 HTTP/1.1" 200 None 2026-03-26 01:15:02,002 - lava_callback - DEBUG - Processing node 69c4884f590270ad7b6d9510: baseline-arm64 2026-03-26 01:15:02,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:02,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:02,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:02,145 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c48461590270ad7b6d93d6/lava-logs.yaml 2026-03-26 01:15:02,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48857590270ad7b6d954a/log.txt.gz 2026-03-26 01:15:02,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:02,403 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48850590270ad7b6d9513/lava_callback.json.gz 2026-03-26 01:15:02,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:02,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:02,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48850590270ad7b6d9513 HTTP/1.1" 200 None 2026-03-26 01:15:02,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:02,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:02,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:02,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:02,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:02,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48850590270ad7b6d9513 HTTP/1.1" 200 None 2026-03-26 01:15:02,758 - lava_callback - INFO - Completed processing callback for node 69c48850590270ad7b6d9513 2026-03-26 01:15:02,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:02,854 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c48461590270ad7b6d93d6/lava_callback.json.gz 2026-03-26 01:15:03,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:03,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:03,017 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48857590270ad7b6d954a/lava-logs.yaml 2026-03-26 01:15:03,028 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884f590270ad7b6d9510/log.txt.gz 2026-03-26 01:15:03,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:03,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48461590270ad7b6d93d6 HTTP/1.1" 200 None 2026-03-26 01:15:03,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b41590270ad7b6d6994 HTTP/1.1" 200 None 2026-03-26 01:15:03,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:03,355 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48857590270ad7b6d954a/lava_callback.json.gz 2026-03-26 01:15:03,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48461590270ad7b6d93d6 HTTP/1.1" 200 None 2026-03-26 01:15:03,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:03,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48857590270ad7b6d954a HTTP/1.1" 200 None 2026-03-26 01:15:03,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,576 - lava_callback - INFO - Completed processing callback for node 69c48461590270ad7b6d93d6 2026-03-26 01:15:03,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:03,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48857590270ad7b6d954a HTTP/1.1" 200 None 2026-03-26 01:15:03,644 - lava_callback - INFO - Completed processing callback for node 69c48857590270ad7b6d954a 2026-03-26 01:15:03,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:03,789 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884f590270ad7b6d9510/lava-logs.yaml 2026-03-26 01:15:03,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:03,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:03,932 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884f590270ad7b6d9510/lava_callback.json.gz 2026-03-26 01:15:03,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884f590270ad7b6d9510 HTTP/1.1" 200 None 2026-03-26 01:15:03,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:03,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:03,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:04,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4884f590270ad7b6d9510 HTTP/1.1" 200 None 2026-03-26 01:15:04,016 - lava_callback - INFO - Completed processing callback for node 69c4884f590270ad7b6d9510 2026-03-26 01:15:04,152 - lava_callback - INFO - Received callback for node 69c48853590270ad7b6d951c INFO: 172.21.0.1:36586 - "POST /node/69c48853590270ad7b6d951c HTTP/1.0" 202 Accepted 2026-03-26 01:15:04,159 - lava_callback - INFO - Starting async job submit for node 69c48853590270ad7b6d951c 2026-03-26 01:15:04,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:04,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48853590270ad7b6d951c HTTP/1.1" 200 None 2026-03-26 01:15:04,213 - lava_callback - DEBUG - Processing node 69c48853590270ad7b6d951c: baseline-arm64 2026-03-26 01:15:04,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:04,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:04,541 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48853590270ad7b6d951c/log.txt.gz 2026-03-26 01:15:04,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:04,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:04,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48853590270ad7b6d951c/lava-logs.yaml 2026-03-26 01:15:04,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:05,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:05,056 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48853590270ad7b6d951c/lava_callback.json.gz 2026-03-26 01:15:05,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:05,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48853590270ad7b6d951c HTTP/1.1" 200 None 2026-03-26 01:15:05,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:05,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:05,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:05,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48853590270ad7b6d951c HTTP/1.1" 200 None 2026-03-26 01:15:05,150 - lava_callback - INFO - Completed processing callback for node 69c48853590270ad7b6d951c INFO: 172.21.0.1:36588 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:15:09,024 - lava_callback - INFO - Received callback for node 69c48855590270ad7b6d9521 INFO: 172.21.0.1:36590 - "POST /node/69c48855590270ad7b6d9521 HTTP/1.0" 202 Accepted 2026-03-26 01:15:09,030 - lava_callback - INFO - Starting async job submit for node 69c48855590270ad7b6d9521 2026-03-26 01:15:09,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:09,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48855590270ad7b6d9521 HTTP/1.1" 200 None 2026-03-26 01:15:09,071 - lava_callback - DEBUG - Processing node 69c48855590270ad7b6d9521: baseline-arm64 2026-03-26 01:15:09,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:09,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:09,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48855590270ad7b6d9521/log.txt.gz 2026-03-26 01:15:09,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:09,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:09,845 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48855590270ad7b6d9521/lava-logs.yaml 2026-03-26 01:15:09,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:09,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:09,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48855590270ad7b6d9521/lava_callback.json.gz 2026-03-26 01:15:10,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:10,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48855590270ad7b6d9521 HTTP/1.1" 200 None 2026-03-26 01:15:10,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:10,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:10,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:10,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48855590270ad7b6d9521 HTTP/1.1" 200 None 2026-03-26 01:15:10,076 - lava_callback - INFO - Completed processing callback for node 69c48855590270ad7b6d9521 INFO: 172.21.0.1:40406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:15:20,552 - lava_callback - INFO - Received callback for node 69c48849590270ad7b6d9501 INFO: 172.21.0.1:37680 - "POST /node/69c48849590270ad7b6d9501 HTTP/1.0" 202 Accepted 2026-03-26 01:15:20,558 - lava_callback - INFO - Starting async job submit for node 69c48849590270ad7b6d9501 2026-03-26 01:15:20,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:20,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48849590270ad7b6d9501 HTTP/1.1" 200 None 2026-03-26 01:15:20,595 - lava_callback - DEBUG - Processing node 69c48849590270ad7b6d9501: baseline-arm64 2026-03-26 01:15:20,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:20,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:20,946 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48849590270ad7b6d9501/log.txt.gz 2026-03-26 01:15:21,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:21,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:21,261 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48849590270ad7b6d9501/lava-logs.yaml 2026-03-26 01:15:21,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:21,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:21,420 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48849590270ad7b6d9501/lava_callback.json.gz 2026-03-26 01:15:21,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:21,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48849590270ad7b6d9501 HTTP/1.1" 200 None 2026-03-26 01:15:21,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:21,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:21,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:21,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48849590270ad7b6d9501 HTTP/1.1" 200 None 2026-03-26 01:15:21,506 - lava_callback - INFO - Completed processing callback for node 69c48849590270ad7b6d9501 INFO: 172.21.0.1:37688 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:15:29,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:29,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 01:15:35,434 - lava_callback - INFO - Received callback for node 69c4884d590270ad7b6d950d INFO: 172.21.0.1:51714 - "POST /node/69c4884d590270ad7b6d950d HTTP/1.0" 202 Accepted 2026-03-26 01:15:35,440 - lava_callback - INFO - Starting async job submit for node 69c4884d590270ad7b6d950d 2026-03-26 01:15:35,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:35,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884d590270ad7b6d950d HTTP/1.1" 200 None 2026-03-26 01:15:35,495 - lava_callback - DEBUG - Processing node 69c4884d590270ad7b6d950d: baseline-arm64 2026-03-26 01:15:35,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:35,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:35,879 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884d590270ad7b6d950d/log.txt.gz 2026-03-26 01:15:36,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:36,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:36,245 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884d590270ad7b6d950d/lava-logs.yaml 2026-03-26 01:15:36,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:36,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:36,417 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884d590270ad7b6d950d/lava_callback.json.gz 2026-03-26 01:15:36,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:36,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884d590270ad7b6d950d HTTP/1.1" 200 None 2026-03-26 01:15:36,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:36,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:36,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:36,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4884d590270ad7b6d950d HTTP/1.1" 200 None 2026-03-26 01:15:36,502 - lava_callback - INFO - Completed processing callback for node 69c4884d590270ad7b6d950d INFO: 172.21.0.1:51724 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:15:37,294 - lava_callback - INFO - Received callback for node 69c48865590270ad7b6d956b INFO: 172.21.0.1:51732 - "POST /node/69c48865590270ad7b6d956b HTTP/1.0" 202 Accepted 2026-03-26 01:15:37,301 - lava_callback - INFO - Starting async job submit for node 69c48865590270ad7b6d956b 2026-03-26 01:15:37,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:37,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48865590270ad7b6d956b HTTP/1.1" 200 None 2026-03-26 01:15:37,338 - lava_callback - DEBUG - Processing node 69c48865590270ad7b6d956b: baseline-nfs-arm64 2026-03-26 01:15:37,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:37,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:37,804 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48865590270ad7b6d956b/log.txt.gz 2026-03-26 01:15:37,975 - lava_callback - INFO - Received callback for node 69c48847590270ad7b6d94fb INFO: 172.21.0.1:51738 - "POST /node/69c48847590270ad7b6d94fb HTTP/1.0" 202 Accepted 2026-03-26 01:15:38,002 - lava_callback - INFO - Starting async job submit for node 69c48847590270ad7b6d94fb 2026-03-26 01:15:38,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:38,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:38,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48847590270ad7b6d94fb HTTP/1.1" 200 None 2026-03-26 01:15:38,078 - lava_callback - DEBUG - Processing node 69c48847590270ad7b6d94fb: baseline-arm64 2026-03-26 01:15:38,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:38,362 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48865590270ad7b6d956b/lava-logs.yaml 2026-03-26 01:15:38,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:38,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:38,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:38,509 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48847590270ad7b6d94fb/log.txt.gz 2026-03-26 01:15:38,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:38,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48865590270ad7b6d956b/lava_callback.json.gz 2026-03-26 01:15:38,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:38,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48865590270ad7b6d956b HTTP/1.1" 200 None 2026-03-26 01:15:38,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:38,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:38,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:38,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:38,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48865590270ad7b6d956b HTTP/1.1" 200 None 2026-03-26 01:15:38,746 - lava_callback - INFO - Completed processing callback for node 69c48865590270ad7b6d956b 2026-03-26 01:15:38,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:38,920 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48847590270ad7b6d94fb/lava-logs.yaml 2026-03-26 01:15:38,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:39,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:39,095 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48847590270ad7b6d94fb/lava_callback.json.gz 2026-03-26 01:15:39,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:39,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48847590270ad7b6d94fb HTTP/1.1" 200 None 2026-03-26 01:15:39,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:39,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:39,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:39,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48847590270ad7b6d94fb HTTP/1.1" 200 None 2026-03-26 01:15:39,181 - lava_callback - INFO - Completed processing callback for node 69c48847590270ad7b6d94fb 2026-03-26 01:15:39,759 - lava_callback - INFO - Received callback for node 69c48864590270ad7b6d9568 INFO: 172.21.0.1:37540 - "POST /node/69c48864590270ad7b6d9568 HTTP/1.0" 202 Accepted 2026-03-26 01:15:39,765 - lava_callback - INFO - Starting async job submit for node 69c48864590270ad7b6d9568 2026-03-26 01:15:39,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:39,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48864590270ad7b6d9568 HTTP/1.1" 200 None 2026-03-26 01:15:39,804 - lava_callback - DEBUG - Processing node 69c48864590270ad7b6d9568: baseline-nfs-arm64 2026-03-26 01:15:39,889 - lava_callback - INFO - Received callback for node 69c48866590270ad7b6d956e INFO: 172.21.0.1:37552 - "POST /node/69c48866590270ad7b6d956e HTTP/1.0" 202 Accepted 2026-03-26 01:15:39,911 - lava_callback - INFO - Starting async job submit for node 69c48866590270ad7b6d956e 2026-03-26 01:15:39,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:40,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48866590270ad7b6d956e HTTP/1.1" 200 None 2026-03-26 01:15:40,075 - lava_callback - DEBUG - Processing node 69c48866590270ad7b6d956e: baseline-nfs-arm64 2026-03-26 01:15:40,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:40,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:40,649 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48864590270ad7b6d9568/log.txt.gz 2026-03-26 01:15:40,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:40,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:41,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:41,018 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48866590270ad7b6d956e/log.txt.gz 2026-03-26 01:15:41,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:41,249 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48864590270ad7b6d9568/lava-logs.yaml 2026-03-26 01:15:41,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:41,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:41,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:41,454 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48864590270ad7b6d9568/lava_callback.json.gz 2026-03-26 01:15:41,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:41,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48864590270ad7b6d9568 HTTP/1.1" 200 None 2026-03-26 01:15:41,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:41,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:41,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:41,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:41,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48866590270ad7b6d956e/lava-logs.yaml 2026-03-26 01:15:41,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:41,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48864590270ad7b6d9568 HTTP/1.1" 200 None 2026-03-26 01:15:41,558 - lava_callback - INFO - Completed processing callback for node 69c48864590270ad7b6d9568 2026-03-26 01:15:41,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:41,742 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48866590270ad7b6d956e/lava_callback.json.gz 2026-03-26 01:15:41,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:41,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48866590270ad7b6d956e HTTP/1.1" 200 None 2026-03-26 01:15:41,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:41,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:41,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:41,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48866590270ad7b6d956e HTTP/1.1" 200 None 2026-03-26 01:15:41,820 - lava_callback - INFO - Completed processing callback for node 69c48866590270ad7b6d956e 2026-03-26 01:15:45,376 - lava_callback - INFO - Received callback for node 69c48862590270ad7b6d9565 INFO: 172.21.0.1:37566 - "POST /node/69c48862590270ad7b6d9565 HTTP/1.0" 202 Accepted 2026-03-26 01:15:45,382 - lava_callback - INFO - Starting async job submit for node 69c48862590270ad7b6d9565 2026-03-26 01:15:45,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:45,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48862590270ad7b6d9565 HTTP/1.1" 200 None 2026-03-26 01:15:45,420 - lava_callback - DEBUG - Processing node 69c48862590270ad7b6d9565: baseline-nfs-arm64 2026-03-26 01:15:45,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:45,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:45,866 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48862590270ad7b6d9565/log.txt.gz 2026-03-26 01:15:46,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:46,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:46,325 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48862590270ad7b6d9565/lava-logs.yaml 2026-03-26 01:15:46,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:46,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:46,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48862590270ad7b6d9565/lava_callback.json.gz 2026-03-26 01:15:46,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:46,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48862590270ad7b6d9565 HTTP/1.1" 200 None 2026-03-26 01:15:46,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:37578 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:15:46,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:46,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:46,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48862590270ad7b6d9565 HTTP/1.1" 200 None 2026-03-26 01:15:46,558 - lava_callback - INFO - Completed processing callback for node 69c48862590270ad7b6d9565 2026-03-26 01:15:48,072 - lava_callback - INFO - Received callback for node 69c4885f590270ad7b6d955c INFO: 172.21.0.1:37584 - "POST /node/69c4885f590270ad7b6d955c HTTP/1.0" 202 Accepted 2026-03-26 01:15:48,082 - lava_callback - INFO - Starting async job submit for node 69c4885f590270ad7b6d955c 2026-03-26 01:15:48,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:48,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885f590270ad7b6d955c HTTP/1.1" 200 None 2026-03-26 01:15:48,136 - lava_callback - DEBUG - Processing node 69c4885f590270ad7b6d955c: baseline-nfs-arm64 2026-03-26 01:15:48,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:48,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:48,804 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885f590270ad7b6d955c/log.txt.gz 2026-03-26 01:15:49,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:49,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:49,331 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885f590270ad7b6d955c/lava-logs.yaml 2026-03-26 01:15:49,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:49,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:49,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885f590270ad7b6d955c/lava_callback.json.gz 2026-03-26 01:15:49,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:49,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885f590270ad7b6d955c HTTP/1.1" 200 None 2026-03-26 01:15:49,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:49,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:49,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4885f590270ad7b6d955c HTTP/1.1" 200 None 2026-03-26 01:15:49,602 - lava_callback - INFO - Completed processing callback for node 69c4885f590270ad7b6d955c 2026-03-26 01:15:52,067 - lava_callback - INFO - Received callback for node 69c48860590270ad7b6d955f INFO: 172.21.0.1:52954 - "POST /node/69c48860590270ad7b6d955f HTTP/1.0" 202 Accepted 2026-03-26 01:15:52,074 - lava_callback - INFO - Starting async job submit for node 69c48860590270ad7b6d955f 2026-03-26 01:15:52,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:52,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48860590270ad7b6d955f HTTP/1.1" 200 None 2026-03-26 01:15:52,112 - lava_callback - DEBUG - Processing node 69c48860590270ad7b6d955f: baseline-nfs-arm64 INFO: 172.21.0.1:52964 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:15:52,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:52,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:52,783 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48860590270ad7b6d955f/log.txt.gz 2026-03-26 01:15:53,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:53,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:53,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48860590270ad7b6d955f/lava-logs.yaml 2026-03-26 01:15:53,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:15:53,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:15:53,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48860590270ad7b6d955f/lava_callback.json.gz 2026-03-26 01:15:53,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:53,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48860590270ad7b6d955f HTTP/1.1" 200 None 2026-03-26 01:15:53,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:53,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:15:53,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:53,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48860590270ad7b6d955f HTTP/1.1" 200 None 2026-03-26 01:15:53,627 - lava_callback - INFO - Completed processing callback for node 69c48860590270ad7b6d955f INFO: 172.21.0.1:52974 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:15:59,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:59,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 01:15:59,795 - lava_callback - INFO - Received callback for node 69c4885d590270ad7b6d9556 INFO: 172.21.0.1:53416 - "POST /node/69c4885d590270ad7b6d9556 HTTP/1.0" 202 Accepted 2026-03-26 01:15:59,802 - lava_callback - INFO - Starting async job submit for node 69c4885d590270ad7b6d9556 2026-03-26 01:15:59,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:15:59,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885d590270ad7b6d9556 HTTP/1.1" 200 None 2026-03-26 01:15:59,849 - lava_callback - DEBUG - Processing node 69c4885d590270ad7b6d9556: baseline-nfs-arm64 2026-03-26 01:16:00,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:00,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:00,333 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885d590270ad7b6d9556/log.txt.gz 2026-03-26 01:16:00,392 - lava_callback - INFO - Received callback for node 69c4885a590270ad7b6d9550 INFO: 172.21.0.1:53418 - "POST /node/69c4885a590270ad7b6d9550 HTTP/1.0" 202 Accepted 2026-03-26 01:16:00,408 - lava_callback - INFO - Starting async job submit for node 69c4885a590270ad7b6d9550 2026-03-26 01:16:00,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:00,468 - lava_callback - INFO - Received callback for node 69c4888b590270ad7b6d95db INFO: 172.21.0.1:53426 - "POST /node/69c4888b590270ad7b6d95db HTTP/1.0" 202 Accepted 2026-03-26 01:16:00,480 - lava_callback - INFO - Starting async job submit for node 69c4888b590270ad7b6d95db 2026-03-26 01:16:00,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:00,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885a590270ad7b6d9550 HTTP/1.1" 200 None 2026-03-26 01:16:00,576 - lava_callback - DEBUG - Processing node 69c4885a590270ad7b6d9550: baseline-nfs-arm64 2026-03-26 01:16:00,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4888b590270ad7b6d95db HTTP/1.1" 200 None 2026-03-26 01:16:00,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:00,725 - lava_callback - DEBUG - Processing node 69c4888b590270ad7b6d95db: baseline-arm64-foundriesio 2026-03-26 01:16:01,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:01,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:01,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:01,286 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4888b590270ad7b6d95db/log.txt.gz 2026-03-26 01:16:01,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:01,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:01,415 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885a590270ad7b6d9550/log.txt.gz 2026-03-26 01:16:01,423 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885d590270ad7b6d9556/lava-logs.yaml 2026-03-26 01:16:01,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:01,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:01,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:01,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:01,721 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4888b590270ad7b6d95db/lava-logs.yaml 2026-03-26 01:16:01,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:01,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:01,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885d590270ad7b6d9556/lava_callback.json.gz 2026-03-26 01:16:01,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:01,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885d590270ad7b6d9556 HTTP/1.1" 200 None 2026-03-26 01:16:01,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:01,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:01,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:01,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4885d590270ad7b6d9556 HTTP/1.1" 200 None 2026-03-26 01:16:01,895 - lava_callback - INFO - Completed processing callback for node 69c4885d590270ad7b6d9556 2026-03-26 01:16:01,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:01,932 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4888b590270ad7b6d95db/lava_callback.json.gz 2026-03-26 01:16:01,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:01,968 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885a590270ad7b6d9550/lava-logs.yaml 2026-03-26 01:16:02,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:02,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:02,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4888b590270ad7b6d95db HTTP/1.1" 200 None 2026-03-26 01:16:02,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:02,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:02,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:02,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4888b590270ad7b6d95db HTTP/1.1" 200 None 2026-03-26 01:16:02,190 - lava_callback - INFO - Completed processing callback for node 69c4888b590270ad7b6d95db 2026-03-26 01:16:02,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:02,303 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885a590270ad7b6d9550/lava_callback.json.gz 2026-03-26 01:16:02,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:02,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885a590270ad7b6d9550 HTTP/1.1" 200 None 2026-03-26 01:16:02,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:02,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:02,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:02,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4885a590270ad7b6d9550 HTTP/1.1" 200 None 2026-03-26 01:16:02,416 - lava_callback - INFO - Completed processing callback for node 69c4885a590270ad7b6d9550 2026-03-26 01:16:05,279 - lava_callback - INFO - Received callback for node 69c48887590270ad7b6d95d2 INFO: 172.21.0.1:53442 - "POST /node/69c48887590270ad7b6d95d2 HTTP/1.0" 202 Accepted 2026-03-26 01:16:05,285 - lava_callback - INFO - Starting async job submit for node 69c48887590270ad7b6d95d2 2026-03-26 01:16:05,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:05,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48887590270ad7b6d95d2 HTTP/1.1" 200 None 2026-03-26 01:16:05,321 - lava_callback - DEBUG - Processing node 69c48887590270ad7b6d95d2: baseline-arm64-pengutronix 2026-03-26 01:16:05,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:05,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:05,611 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c48887590270ad7b6d95d2/log.txt.gz 2026-03-26 01:16:05,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:05,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:05,892 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c48887590270ad7b6d95d2/lava-logs.yaml 2026-03-26 01:16:05,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:06,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:06,031 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c48887590270ad7b6d95d2/lava_callback.json.gz 2026-03-26 01:16:06,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:06,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48887590270ad7b6d95d2 HTTP/1.1" 200 None 2026-03-26 01:16:06,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:06,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:06,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:06,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48887590270ad7b6d95d2 HTTP/1.1" 200 None 2026-03-26 01:16:06,108 - lava_callback - INFO - Completed processing callback for node 69c48887590270ad7b6d95d2 INFO: 172.21.0.1:53458 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:16:12,352 - lava_callback - INFO - Received callback for node 69c4887c590270ad7b6d95b7 INFO: 172.21.0.1:42900 - "POST /node/69c4887c590270ad7b6d95b7 HTTP/1.0" 202 Accepted 2026-03-26 01:16:12,358 - lava_callback - INFO - Starting async job submit for node 69c4887c590270ad7b6d95b7 2026-03-26 01:16:12,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:12,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887c590270ad7b6d95b7 HTTP/1.1" 200 None 2026-03-26 01:16:12,397 - lava_callback - DEBUG - Processing node 69c4887c590270ad7b6d95b7: baseline-arm64-broonie 2026-03-26 01:16:12,475 - lava_callback - INFO - Received callback for node 69c48888590270ad7b6d95d5 INFO: 172.21.0.1:42906 - "POST /node/69c48888590270ad7b6d95d5 HTTP/1.0" 202 Accepted 2026-03-26 01:16:12,497 - lava_callback - INFO - Starting async job submit for node 69c48888590270ad7b6d95d5 2026-03-26 01:16:12,532 - lava_callback - INFO - Received callback for node 69c48851590270ad7b6d9516 2026-03-26 01:16:12,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42922 - "POST /node/69c48851590270ad7b6d9516 HTTP/1.0" 202 Accepted 2026-03-26 01:16:12,555 - lava_callback - INFO - Starting async job submit for node 69c48851590270ad7b6d9516 2026-03-26 01:16:12,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:12,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48888590270ad7b6d95d5 HTTP/1.1" 200 None 2026-03-26 01:16:12,687 - lava_callback - DEBUG - Processing node 69c48888590270ad7b6d95d5: baseline-arm64-pengutronix 2026-03-26 01:16:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48851590270ad7b6d9516 HTTP/1.1" 200 None 2026-03-26 01:16:12,735 - lava_callback - DEBUG - Processing node 69c48851590270ad7b6d9516: baseline-arm64 2026-03-26 01:16:13,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:13,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:13,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:13,369 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887c590270ad7b6d95b7/log.txt.gz 2026-03-26 01:16:13,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:13,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:13,446 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c48888590270ad7b6d95d5/log.txt.gz 2026-03-26 01:16:13,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:13,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:13,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:13,782 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48851590270ad7b6d9516/log.txt.gz 2026-03-26 01:16:13,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:13,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887c590270ad7b6d95b7/lava-logs.yaml 2026-03-26 01:16:13,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:13,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c48888590270ad7b6d95d5/lava-logs.yaml 2026-03-26 01:16:13,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:14,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:14,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:14,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:14,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:14,262 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887c590270ad7b6d95b7/lava_callback.json.gz 2026-03-26 01:16:14,272 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c48888590270ad7b6d95d5/lava_callback.json.gz 2026-03-26 01:16:14,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:14,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,315 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48851590270ad7b6d9516/lava-logs.yaml 2026-03-26 01:16:14,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:14,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887c590270ad7b6d95b7 HTTP/1.1" 200 None 2026-03-26 01:16:14,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48888590270ad7b6d95d5 HTTP/1.1" 200 None 2026-03-26 01:16:14,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:14,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:14,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48888590270ad7b6d95d5 HTTP/1.1" 200 None 2026-03-26 01:16:14,408 - lava_callback - INFO - Completed processing callback for node 69c48888590270ad7b6d95d5 2026-03-26 01:16:14,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4887c590270ad7b6d95b7 HTTP/1.1" 200 None 2026-03-26 01:16:14,433 - lava_callback - INFO - Completed processing callback for node 69c4887c590270ad7b6d95b7 2026-03-26 01:16:14,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:14,586 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48851590270ad7b6d9516/lava_callback.json.gz 2026-03-26 01:16:14,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48851590270ad7b6d9516 HTTP/1.1" 200 None 2026-03-26 01:16:14,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:14,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:14,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48851590270ad7b6d9516 HTTP/1.1" 200 None 2026-03-26 01:16:14,766 - lava_callback - INFO - Completed processing callback for node 69c48851590270ad7b6d9516 INFO: 172.21.0.1:42934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42936 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:16:18,918 - lava_callback - INFO - Received callback for node 69c4888a590270ad7b6d95d8 INFO: 172.21.0.1:42952 - "POST /node/69c4888a590270ad7b6d95d8 HTTP/1.0" 202 Accepted 2026-03-26 01:16:18,924 - lava_callback - INFO - Starting async job submit for node 69c4888a590270ad7b6d95d8 2026-03-26 01:16:18,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:18,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4888a590270ad7b6d95d8 HTTP/1.1" 200 None 2026-03-26 01:16:18,980 - lava_callback - DEBUG - Processing node 69c4888a590270ad7b6d95d8: baseline-arm64-pengutronix 2026-03-26 01:16:19,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:19,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:19,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4888a590270ad7b6d95d8/log.txt.gz 2026-03-26 01:16:19,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:19,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:19,623 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4888a590270ad7b6d95d8/lava-logs.yaml 2026-03-26 01:16:19,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:19,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:19,770 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c4888a590270ad7b6d95d8/lava_callback.json.gz 2026-03-26 01:16:19,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:19,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4888a590270ad7b6d95d8 HTTP/1.1" 200 None 2026-03-26 01:16:19,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:19,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:19,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:19,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4888a590270ad7b6d95d8 HTTP/1.1" 200 None 2026-03-26 01:16:19,854 - lava_callback - INFO - Completed processing callback for node 69c4888a590270ad7b6d95d8 INFO: 172.21.0.1:41692 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:16:29,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:29,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-26 01:16:30,491 - lava_callback - INFO - Received callback for node 69c48874590270ad7b6d958a INFO: 172.21.0.1:60024 - "POST /node/69c48874590270ad7b6d958a HTTP/1.0" 202 Accepted 2026-03-26 01:16:30,497 - lava_callback - INFO - Starting async job submit for node 69c48874590270ad7b6d958a 2026-03-26 01:16:30,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:30,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48874590270ad7b6d958a HTTP/1.1" 200 None 2026-03-26 01:16:30,537 - lava_callback - DEBUG - Processing node 69c48874590270ad7b6d958a: baseline-arm64-broonie 2026-03-26 01:16:30,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:30,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:30,904 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48874590270ad7b6d958a/log.txt.gz 2026-03-26 01:16:31,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:31,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:31,257 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48874590270ad7b6d958a/lava-logs.yaml 2026-03-26 01:16:31,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:31,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:31,407 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48874590270ad7b6d958a/lava_callback.json.gz 2026-03-26 01:16:31,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:31,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48874590270ad7b6d958a HTTP/1.1" 200 None 2026-03-26 01:16:31,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:31,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:31,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:31,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48874590270ad7b6d958a HTTP/1.1" 200 None 2026-03-26 01:16:31,493 - lava_callback - INFO - Completed processing callback for node 69c48874590270ad7b6d958a INFO: 172.21.0.1:60034 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:16:36,673 - lava_callback - INFO - Received callback for node 69c48883590270ad7b6d95c9 INFO: 172.21.0.1:60044 - "POST /node/69c48883590270ad7b6d95c9 HTTP/1.0" 202 Accepted 2026-03-26 01:16:36,679 - lava_callback - INFO - Starting async job submit for node 69c48883590270ad7b6d95c9 2026-03-26 01:16:36,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:36,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48883590270ad7b6d95c9 HTTP/1.1" 200 None 2026-03-26 01:16:36,763 - lava_callback - DEBUG - Processing node 69c48883590270ad7b6d95c9: baseline-arm64-clabbe 2026-03-26 01:16:37,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:37,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:37,129 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c48883590270ad7b6d95c9/log.txt.gz 2026-03-26 01:16:37,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:37,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:37,440 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c48883590270ad7b6d95c9/lava-logs.yaml 2026-03-26 01:16:37,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:37,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:37,584 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c48883590270ad7b6d95c9/lava_callback.json.gz 2026-03-26 01:16:37,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:37,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48883590270ad7b6d95c9 HTTP/1.1" 200 None 2026-03-26 01:16:37,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:37,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:37,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:37,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48883590270ad7b6d95c9 HTTP/1.1" 200 None 2026-03-26 01:16:37,679 - lava_callback - INFO - Completed processing callback for node 69c48883590270ad7b6d95c9 2026-03-26 01:16:39,478 - lava_callback - INFO - Received callback for node 69c4887d590270ad7b6d95ba INFO: 172.21.0.1:60060 - "POST /node/69c4887d590270ad7b6d95ba HTTP/1.0" 202 Accepted 2026-03-26 01:16:39,484 - lava_callback - INFO - Starting async job submit for node 69c4887d590270ad7b6d95ba 2026-03-26 01:16:39,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:39,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887d590270ad7b6d95ba HTTP/1.1" 200 None 2026-03-26 01:16:39,523 - lava_callback - DEBUG - Processing node 69c4887d590270ad7b6d95ba: baseline-arm64-broonie 2026-03-26 01:16:39,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:39,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:39,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887d590270ad7b6d95ba/log.txt.gz 2026-03-26 01:16:40,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:40,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:40,233 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887d590270ad7b6d95ba/lava-logs.yaml 2026-03-26 01:16:40,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:16:40,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:16:40,372 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887d590270ad7b6d95ba/lava_callback.json.gz 2026-03-26 01:16:40,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:40,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887d590270ad7b6d95ba HTTP/1.1" 200 None 2026-03-26 01:16:40,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:40,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:16:40,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:40,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4887d590270ad7b6d95ba HTTP/1.1" 200 None 2026-03-26 01:16:40,470 - lava_callback - INFO - Completed processing callback for node 69c4887d590270ad7b6d95ba INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53044 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:16:59,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:16:59,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 01:17:06,044 - lava_callback - INFO - Received callback for node 69c4884c590270ad7b6d950a INFO: 172.21.0.1:42872 - "POST /node/69c4884c590270ad7b6d950a HTTP/1.0" 202 Accepted 2026-03-26 01:17:06,049 - lava_callback - INFO - Starting async job submit for node 69c4884c590270ad7b6d950a 2026-03-26 01:17:06,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:17:06,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884c590270ad7b6d950a HTTP/1.1" 200 None 2026-03-26 01:17:06,082 - lava_callback - DEBUG - Processing node 69c4884c590270ad7b6d950a: baseline-arm64 2026-03-26 01:17:06,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42874 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:17:06,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:17:06,755 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884c590270ad7b6d950a/log.txt.gz 2026-03-26 01:17:06,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:17:07,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:17:07,161 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884c590270ad7b6d950a/lava-logs.yaml 2026-03-26 01:17:07,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:17:07,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:17:07,622 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4884c590270ad7b6d950a/lava_callback.json.gz 2026-03-26 01:17:07,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:17:07,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4884c590270ad7b6d950a HTTP/1.1" 200 None 2026-03-26 01:17:07,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:17:07,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:17:07,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:17:07,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4884c590270ad7b6d950a HTTP/1.1" 200 None 2026-03-26 01:17:07,698 - lava_callback - INFO - Completed processing callback for node 69c4884c590270ad7b6d950a INFO: 172.21.0.1:32918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:17:29,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:17:29,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42486 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:17:59,808 - lava_callback - INFO - Received callback for node 69c4888c590270ad7b6d95de INFO: 172.21.0.1:48886 - "POST /node/69c4888c590270ad7b6d95de HTTP/1.0" 202 Accepted 2026-03-26 01:17:59,816 - lava_callback - INFO - Starting async job submit for node 69c4888c590270ad7b6d95de 2026-03-26 01:17:59,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:17:59,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4888c590270ad7b6d95de HTTP/1.1" 200 None 2026-03-26 01:17:59,868 - lava_callback - DEBUG - Processing node 69c4888c590270ad7b6d95de: baseline-arm64-foundriesio 2026-03-26 01:18:00,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:00,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:00,855 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4888c590270ad7b6d95de/log.txt.gz 2026-03-26 01:18:01,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:01,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:01,576 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4888c590270ad7b6d95de/lava-logs.yaml 2026-03-26 01:18:01,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:01,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:01,786 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c4888c590270ad7b6d95de/lava_callback.json.gz DEBUG: 69c4888c590270ad7b6d95de Transitting job node result: pass -> incomplete 2026-03-26 01:18:01,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:01,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4888c590270ad7b6d95de HTTP/1.1" 200 None 2026-03-26 01:18:01,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:01,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:18:01,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:01,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4888c590270ad7b6d95de HTTP/1.1" 200 None 2026-03-26 01:18:01,888 - lava_callback - INFO - Completed processing callback for node 69c4888c590270ad7b6d95de INFO: 172.21.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:18:19,503 - lava_callback - INFO - Received callback for node 69c48876590270ad7b6d95aa INFO: 172.21.0.1:39650 - "POST /node/69c48876590270ad7b6d95aa HTTP/1.0" 202 Accepted 2026-03-26 01:18:19,510 - lava_callback - INFO - Starting async job submit for node 69c48876590270ad7b6d95aa 2026-03-26 01:18:19,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:19,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48876590270ad7b6d95aa HTTP/1.1" 200 None 2026-03-26 01:18:19,547 - lava_callback - DEBUG - Processing node 69c48876590270ad7b6d95aa: baseline-arm64-broonie 2026-03-26 01:18:19,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:20,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:20,103 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48876590270ad7b6d95aa/log.txt.gz 2026-03-26 01:18:20,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:20,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:20,489 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48876590270ad7b6d95aa/lava-logs.yaml 2026-03-26 01:18:20,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:20,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:20,665 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48876590270ad7b6d95aa/lava_callback.json.gz 2026-03-26 01:18:20,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:20,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48876590270ad7b6d95aa HTTP/1.1" 200 None 2026-03-26 01:18:20,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:20,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:18:20,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:20,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48876590270ad7b6d95aa HTTP/1.1" 200 None 2026-03-26 01:18:20,749 - lava_callback - INFO - Completed processing callback for node 69c48876590270ad7b6d95aa INFO: 172.21.0.1:51824 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:18:29,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:29,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 01:18:30,594 - lava_callback - INFO - Received callback for node 69c4885b590270ad7b6d9553 INFO: 172.21.0.1:45612 - "POST /node/69c4885b590270ad7b6d9553 HTTP/1.0" 202 Accepted 2026-03-26 01:18:30,601 - lava_callback - INFO - Starting async job submit for node 69c4885b590270ad7b6d9553 2026-03-26 01:18:30,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:30,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885b590270ad7b6d9553 HTTP/1.1" 200 None 2026-03-26 01:18:30,636 - lava_callback - DEBUG - Processing node 69c4885b590270ad7b6d9553: baseline-nfs-arm64 2026-03-26 01:18:31,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:31,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:31,207 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885b590270ad7b6d9553/log.txt.gz 2026-03-26 01:18:31,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:31,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:31,666 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885b590270ad7b6d9553/lava-logs.yaml 2026-03-26 01:18:31,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:18:31,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:18:31,860 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885b590270ad7b6d9553/lava_callback.json.gz 2026-03-26 01:18:31,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:31,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885b590270ad7b6d9553 HTTP/1.1" 200 None 2026-03-26 01:18:31,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:31,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:18:31,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:31,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4885b590270ad7b6d9553 HTTP/1.1" 200 None 2026-03-26 01:18:31,944 - lava_callback - INFO - Completed processing callback for node 69c4885b590270ad7b6d9553 INFO: 172.21.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:18:59,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:18:59,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54360 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:19:51,623 - lava_callback - INFO - Received callback for node 69c1e6a20efaf21d93106429 INFO: 172.21.0.1:48718 - "POST /node/69c1e6a20efaf21d93106429 HTTP/1.0" 202 Accepted 2026-03-26 01:19:51,628 - lava_callback - INFO - Starting async job submit for node 69c1e6a20efaf21d93106429 2026-03-26 01:19:51,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:19:51,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6a20efaf21d93106429 HTTP/1.1" 200 None 2026-03-26 01:19:51,656 - lava_callback - DEBUG - Processing node 69c1e6a20efaf21d93106429: baseline-arm64-baylibre 2026-03-26 01:19:51,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e6a20efaf21d93106429 2026-03-26 01:19:51,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:19:51,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6a20efaf21d93106429 HTTP/1.1" 200 None 2026-03-26 01:19:51,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:19:51,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:19:51,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:19:51,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e6a20efaf21d93106429 HTTP/1.1" 200 None 2026-03-26 01:19:51,713 - lava_callback - ERROR - Error processing callback for node 69c1e6a20efaf21d93106429: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58436 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:21:09,211 - lava_callback - INFO - Received callback for node 69c48621590270ad7b6d9475 INFO: 172.21.0.1:58446 - "POST /node/69c48621590270ad7b6d9475 HTTP/1.0" 202 Accepted 2026-03-26 01:21:09,218 - lava_callback - INFO - Starting async job submit for node 69c48621590270ad7b6d9475 2026-03-26 01:21:09,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:09,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48621590270ad7b6d9475 HTTP/1.1" 200 None 2026-03-26 01:21:09,289 - lava_callback - DEBUG - Processing node 69c48621590270ad7b6d9475: baseline-arm 2026-03-26 01:21:09,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:21:10,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:21:10,042 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c48621590270ad7b6d9475/log.txt.gz 2026-03-26 01:21:10,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:21:10,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:21:10,629 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c48621590270ad7b6d9475/lava-logs.yaml 2026-03-26 01:21:10,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:21:10,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:21:10,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c48621590270ad7b6d9475/lava_callback.json.gz DEBUG: Setup failed for 69c48621590270ad7b6d9475. Transitting job node result: incomplete -> fail 2026-03-26 01:21:10,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:10,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48621590270ad7b6d9475 HTTP/1.1" 200 None 2026-03-26 01:21:10,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:10,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 01:21:10,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:11,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48621590270ad7b6d9475 HTTP/1.1" 200 None 2026-03-26 01:21:11,114 - lava_callback - INFO - Completed processing callback for node 69c48621590270ad7b6d9475 2026-03-26 01:21:11,574 - lava_callback - INFO - Received callback for node 69c1e6fe0efaf21d93106475 INFO: 172.21.0.1:51154 - "POST /node/69c1e6fe0efaf21d93106475 HTTP/1.0" 202 Accepted 2026-03-26 01:21:11,579 - lava_callback - INFO - Starting async job submit for node 69c1e6fe0efaf21d93106475 2026-03-26 01:21:11,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:11,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6fe0efaf21d93106475 HTTP/1.1" 200 None 2026-03-26 01:21:11,605 - lava_callback - DEBUG - Processing node 69c1e6fe0efaf21d93106475: baseline-arm64-baylibre 2026-03-26 01:21:11,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e6fe0efaf21d93106475 2026-03-26 01:21:11,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:11,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e6fe0efaf21d93106475 HTTP/1.1" 200 None 2026-03-26 01:21:11,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:11,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:21:11,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:11,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e6fe0efaf21d93106475 HTTP/1.1" 200 None 2026-03-26 01:21:11,661 - lava_callback - ERROR - Error processing callback for node 69c1e6fe0efaf21d93106475: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57878 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:21:29,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:21:29,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:22:31,747 - lava_callback - INFO - Received callback for node 69c48858590270ad7b6d954d INFO: 172.21.0.1:43376 - "POST /node/69c48858590270ad7b6d954d HTTP/1.0" 202 Accepted 2026-03-26 01:22:31,754 - lava_callback - INFO - Starting async job submit for node 69c48858590270ad7b6d954d 2026-03-26 01:22:31,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:31,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48858590270ad7b6d954d HTTP/1.1" 200 None 2026-03-26 01:22:31,804 - lava_callback - DEBUG - Processing node 69c48858590270ad7b6d954d: baseline-arm64 2026-03-26 01:22:32,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:22:32,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:22:32,347 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48858590270ad7b6d954d/log.txt.gz 2026-03-26 01:22:32,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:22:32,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:22:32,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48858590270ad7b6d954d/lava-logs.yaml 2026-03-26 01:22:32,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:22:32,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:22:32,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48858590270ad7b6d954d/lava_callback.json.gz DEBUG: Setup failed for 69c48858590270ad7b6d954d. Transitting job node result: incomplete -> fail 2026-03-26 01:22:33,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:33,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48858590270ad7b6d954d HTTP/1.1" 200 None 2026-03-26 01:22:33,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:33,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:22:33,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:33,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48858590270ad7b6d954d HTTP/1.1" 200 None 2026-03-26 01:22:33,206 - lava_callback - INFO - Completed processing callback for node 69c48858590270ad7b6d954d INFO: 172.21.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:22:51,689 - lava_callback - INFO - Received callback for node 69c1e7570efaf21d931064c1 INFO: 172.21.0.1:53014 - "POST /node/69c1e7570efaf21d931064c1 HTTP/1.0" 202 Accepted 2026-03-26 01:22:51,695 - lava_callback - INFO - Starting async job submit for node 69c1e7570efaf21d931064c1 2026-03-26 01:22:51,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:51,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e7570efaf21d931064c1 HTTP/1.1" 200 None 2026-03-26 01:22:51,711 - lava_callback - DEBUG - Processing node 69c1e7570efaf21d931064c1: baseline-arm64-baylibre 2026-03-26 01:22:51,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e7570efaf21d931064c1 2026-03-26 01:22:51,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:51,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e7570efaf21d931064c1 HTTP/1.1" 200 None 2026-03-26 01:22:51,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:51,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:22:51,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:51,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e7570efaf21d931064c1 HTTP/1.1" 200 None 2026-03-26 01:22:51,768 - lava_callback - ERROR - Error processing callback for node 69c1e7570efaf21d931064c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53038 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:22:59,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:22:59,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41262 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:24:20,221 - lava_callback - INFO - Received callback for node 69c1e7b50efaf21d9310650e INFO: 172.21.0.1:39320 - "POST /node/69c1e7b50efaf21d9310650e HTTP/1.0" 202 Accepted 2026-03-26 01:24:20,226 - lava_callback - INFO - Starting async job submit for node 69c1e7b50efaf21d9310650e 2026-03-26 01:24:20,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:24:20,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e7b50efaf21d9310650e HTTP/1.1" 200 None 2026-03-26 01:24:20,242 - lava_callback - DEBUG - Processing node 69c1e7b50efaf21d9310650e: baseline-arm64-clabbe 2026-03-26 01:24:20,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e7b50efaf21d9310650e 2026-03-26 01:24:20,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:24:20,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e7b50efaf21d9310650e HTTP/1.1" 200 None 2026-03-26 01:24:20,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:24:20,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:24:20,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:24:20,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e7b50efaf21d9310650e HTTP/1.1" 200 None 2026-03-26 01:24:20,292 - lava_callback - ERROR - Error processing callback for node 69c1e7b50efaf21d9310650e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49196 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:25:03,074 - lava_callback - INFO - Received callback for node 69c48650590270ad7b6d9480 INFO: 172.21.0.1:36990 - "POST /node/69c48650590270ad7b6d9480 HTTP/1.0" 202 Accepted 2026-03-26 01:25:03,121 - lava_callback - INFO - Starting async job submit for node 69c48650590270ad7b6d9480 2026-03-26 01:25:03,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:03,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48650590270ad7b6d9480 HTTP/1.1" 200 None 2026-03-26 01:25:03,160 - lava_callback - DEBUG - Processing node 69c48650590270ad7b6d9480: baseline-arm-pengutronix 2026-03-26 01:25:03,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:25:03,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:25:03,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48650590270ad7b6d9480/log.txt.gz 2026-03-26 01:25:03,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:25:03,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:25:03,674 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48650590270ad7b6d9480/lava-logs.yaml 2026-03-26 01:25:03,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:25:03,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:25:03,978 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48650590270ad7b6d9480/lava_callback.json.gz DEBUG: Setup failed for 69c48650590270ad7b6d9480. Transitting job node result: incomplete -> fail 2026-03-26 01:25:04,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:04,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48650590270ad7b6d9480 HTTP/1.1" 200 None 2026-03-26 01:25:04,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:04,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 01:25:04,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:04,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48650590270ad7b6d9480 HTTP/1.1" 200 None 2026-03-26 01:25:04,150 - lava_callback - INFO - Completed processing callback for node 69c48650590270ad7b6d9480 INFO: 172.21.0.1:37004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:25:29,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:29,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 01:25:33,628 - lava_callback - INFO - Received callback for node 69c48afb590270ad7b6d9773 INFO: 172.21.0.1:58526 - "POST /node/69c48afb590270ad7b6d9773 HTTP/1.0" 202 Accepted 2026-03-26 01:25:33,634 - lava_callback - INFO - Starting async job submit for node 69c48afb590270ad7b6d9773 2026-03-26 01:25:33,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:33,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48afb590270ad7b6d9773 HTTP/1.1" 200 None 2026-03-26 01:25:33,651 - lava_callback - DEBUG - Processing node 69c48afb590270ad7b6d9773: baseline-nfs-arm64 2026-03-26 01:25:33,653 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:25:33,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:25:33,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:25:33,776 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48afb590270ad7b6d9773/lava-logs.yaml 2026-03-26 01:25:33,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:25:34,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:25:34,064 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48afb590270ad7b6d9773/lava_callback.json.gz 2026-03-26 01:25:34,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:34,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48afb590270ad7b6d9773 HTTP/1.1" 200 None 2026-03-26 01:25:34,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:34,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:25:34,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:34,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48afb590270ad7b6d9773 HTTP/1.1" 200 None 2026-03-26 01:25:34,132 - lava_callback - INFO - Completed processing callback for node 69c48afb590270ad7b6d9773 INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:25:40,226 - lava_callback - INFO - Received callback for node 69c1e80d0efaf21d93106551 INFO: 172.21.0.1:47112 - "POST /node/69c1e80d0efaf21d93106551 HTTP/1.0" 202 Accepted 2026-03-26 01:25:40,231 - lava_callback - INFO - Starting async job submit for node 69c1e80d0efaf21d93106551 2026-03-26 01:25:40,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:40,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e80d0efaf21d93106551 HTTP/1.1" 200 None 2026-03-26 01:25:40,250 - lava_callback - DEBUG - Processing node 69c1e80d0efaf21d93106551: baseline-arm64-clabbe 2026-03-26 01:25:40,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e80d0efaf21d93106551 2026-03-26 01:25:40,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:40,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e80d0efaf21d93106551 HTTP/1.1" 200 None 2026-03-26 01:25:40,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:40,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:25:40,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:40,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e80d0efaf21d93106551 HTTP/1.1" 200 None 2026-03-26 01:25:40,310 - lava_callback - ERROR - Error processing callback for node 69c1e80d0efaf21d93106551: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37598 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:25:59,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:25:59,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40942 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:26:30,861 - lava_callback - INFO - Received callback for node 69c48a4b590270ad7b6d9741 INFO: 172.21.0.1:42322 - "POST /node/69c48a4b590270ad7b6d9741 HTTP/1.0" 202 Accepted 2026-03-26 01:26:30,867 - lava_callback - INFO - Starting async job submit for node 69c48a4b590270ad7b6d9741 2026-03-26 01:26:30,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:26:30,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48a4b590270ad7b6d9741 HTTP/1.1" 200 None 2026-03-26 01:26:30,909 - lava_callback - DEBUG - Processing node 69c48a4b590270ad7b6d9741: ltp-input 2026-03-26 01:26:31,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:26:31,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:26:31,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c48a4b590270ad7b6d9741/log.txt.gz 2026-03-26 01:26:31,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:26:31,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:26:31,862 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c48a4b590270ad7b6d9741/lava-logs.yaml 2026-03-26 01:26:31,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:26:32,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:26:32,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c48a4b590270ad7b6d9741/lava_callback.json.gz 2026-03-26 01:26:32,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:26:32,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48a4b590270ad7b6d9741 HTTP/1.1" 200 None 2026-03-26 01:26:32,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:26:32,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 01:26:32,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:26:32,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48a4b590270ad7b6d9741 HTTP/1.1" 200 None 2026-03-26 01:26:32,146 - lava_callback - INFO - Completed processing callback for node 69c48a4b590270ad7b6d9741 INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:26:59,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:26:59,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38486 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:27:22,989 - lava_callback - INFO - Received callback for node 69c48b44590270ad7b6d9784 INFO: 172.21.0.1:43760 - "POST /node/69c48b44590270ad7b6d9784 HTTP/1.0" 202 Accepted 2026-03-26 01:27:22,995 - lava_callback - INFO - Starting async job submit for node 69c48b44590270ad7b6d9784 2026-03-26 01:27:23,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:27:23,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48b44590270ad7b6d9784 HTTP/1.1" 200 None 2026-03-26 01:27:23,041 - lava_callback - DEBUG - Processing node 69c48b44590270ad7b6d9784: baseline-arm64 2026-03-26 01:27:23,171 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:27:23,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:27:23,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:27:23,443 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48b44590270ad7b6d9784/lava-logs.yaml 2026-03-26 01:27:23,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:27:23,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:27:23,574 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48b44590270ad7b6d9784/lava_callback.json.gz 2026-03-26 01:27:23,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:27:23,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48b44590270ad7b6d9784 HTTP/1.1" 200 None 2026-03-26 01:27:23,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:27:23,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:27:23,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:27:23,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48b44590270ad7b6d9784 HTTP/1.1" 200 None 2026-03-26 01:27:23,726 - lava_callback - INFO - Completed processing callback for node 69c48b44590270ad7b6d9784 INFO: 172.21.0.1:43764 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:27:29,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:27:29,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33308 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:28:40,356 - lava_callback - INFO - Received callback for node 69c1e8bb0efaf21d931065c0 INFO: 172.21.0.1:51038 - "POST /node/69c1e8bb0efaf21d931065c0 HTTP/1.0" 202 Accepted 2026-03-26 01:28:40,361 - lava_callback - INFO - Starting async job submit for node 69c1e8bb0efaf21d931065c0 2026-03-26 01:28:40,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:40,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e8bb0efaf21d931065c0 HTTP/1.1" 200 None 2026-03-26 01:28:40,377 - lava_callback - DEBUG - Processing node 69c1e8bb0efaf21d931065c0: baseline-arm64-clabbe 2026-03-26 01:28:40,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e8bb0efaf21d931065c0 2026-03-26 01:28:40,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:40,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e8bb0efaf21d931065c0 HTTP/1.1" 200 None 2026-03-26 01:28:40,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:40,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:28:40,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:40,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e8bb0efaf21d931065c0 HTTP/1.1" 200 None 2026-03-26 01:28:40,432 - lava_callback - ERROR - Error processing callback for node 69c1e8bb0efaf21d931065c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-26 01:28:46,480 - lava_callback - INFO - Received callback for node 69c48b96590270ad7b6d979c INFO: 172.21.0.1:51054 - "POST /node/69c48b96590270ad7b6d979c HTTP/1.0" 202 Accepted 2026-03-26 01:28:46,486 - lava_callback - INFO - Starting async job submit for node 69c48b96590270ad7b6d979c INFO: 172.21.0.1:51060 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:28:46,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:46,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48b96590270ad7b6d979c HTTP/1.1" 200 None 2026-03-26 01:28:46,534 - lava_callback - DEBUG - Processing node 69c48b96590270ad7b6d979c: baseline-arm64 2026-03-26 01:28:46,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:28:46,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:28:46,854 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48b96590270ad7b6d979c/log.txt.gz 2026-03-26 01:28:46,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:28:47,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:28:47,113 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48b96590270ad7b6d979c/lava-logs.yaml 2026-03-26 01:28:47,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:28:47,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:28:47,250 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48b96590270ad7b6d979c/lava_callback.json.gz 2026-03-26 01:28:47,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:47,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48b96590270ad7b6d979c HTTP/1.1" 200 None 2026-03-26 01:28:47,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:47,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:28:47,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:47,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48b96590270ad7b6d979c HTTP/1.1" 200 None 2026-03-26 01:28:47,405 - lava_callback - INFO - Completed processing callback for node 69c48b96590270ad7b6d979c INFO: 172.21.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49166 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:28:59,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:28:59,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34306 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:30:00,471 - lava_callback - INFO - Received callback for node 69c1e9120efaf21d931065f3 INFO: 172.21.0.1:45538 - "POST /node/69c1e9120efaf21d931065f3 HTTP/1.0" 202 Accepted 2026-03-26 01:30:00,478 - lava_callback - INFO - Starting async job submit for node 69c1e9120efaf21d931065f3 2026-03-26 01:30:00,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:00,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e9120efaf21d931065f3 HTTP/1.1" 200 None 2026-03-26 01:30:00,496 - lava_callback - DEBUG - Processing node 69c1e9120efaf21d931065f3: baseline-arm64-clabbe 2026-03-26 01:30:00,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e9120efaf21d931065f3 2026-03-26 01:30:00,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:00,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e9120efaf21d931065f3 HTTP/1.1" 200 None 2026-03-26 01:30:00,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:00,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:30:00,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:00,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e9120efaf21d931065f3 HTTP/1.1" 200 None 2026-03-26 01:30:00,569 - lava_callback - ERROR - Error processing callback for node 69c1e9120efaf21d931065f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33296 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:30:23,108 - lava_callback - INFO - Received callback for node 69c48bfd590270ad7b6d97ba INFO: 172.21.0.1:49000 - "POST /node/69c48bfd590270ad7b6d97ba HTTP/1.0" 202 Accepted 2026-03-26 01:30:23,114 - lava_callback - INFO - Starting async job submit for node 69c48bfd590270ad7b6d97ba 2026-03-26 01:30:23,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:23,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48bfd590270ad7b6d97ba HTTP/1.1" 200 None 2026-03-26 01:30:23,159 - lava_callback - DEBUG - Processing node 69c48bfd590270ad7b6d97ba: baseline-arm64-foundriesio 2026-03-26 01:30:23,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:30:23,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:30:23,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c48bfd590270ad7b6d97ba/log.txt.gz 2026-03-26 01:30:23,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:30:23,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:30:23,850 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c48bfd590270ad7b6d97ba/lava-logs.yaml 2026-03-26 01:30:23,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:30:24,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:30:24,012 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c48bfd590270ad7b6d97ba/lava_callback.json.gz 2026-03-26 01:30:24,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:24,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48bfd590270ad7b6d97ba HTTP/1.1" 200 None 2026-03-26 01:30:24,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:24,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:30:24,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:24,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48bfd590270ad7b6d97ba HTTP/1.1" 200 None 2026-03-26 01:30:24,105 - lava_callback - INFO - Completed processing callback for node 69c48bfd590270ad7b6d97ba INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:30:29,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:30:30,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52668 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:31:36,335 - lava_callback - INFO - Received callback for node 69c48c37590270ad7b6d97c9 INFO: 172.21.0.1:44670 - "POST /node/69c48c37590270ad7b6d97c9 HTTP/1.0" 202 Accepted 2026-03-26 01:31:36,341 - lava_callback - INFO - Starting async job submit for node 69c48c37590270ad7b6d97c9 2026-03-26 01:31:36,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:36,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48c37590270ad7b6d97c9 HTTP/1.1" 200 None 2026-03-26 01:31:36,377 - lava_callback - DEBUG - Processing node 69c48c37590270ad7b6d97c9: baseline-arm64 INFO: 172.21.0.1:44684 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:31:36,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:31:37,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:31:37,081 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48c37590270ad7b6d97c9/log.txt.gz 2026-03-26 01:31:37,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:31:37,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:31:37,546 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48c37590270ad7b6d97c9/lava-logs.yaml 2026-03-26 01:31:37,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:31:37,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:31:37,733 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48c37590270ad7b6d97c9/lava_callback.json.gz 2026-03-26 01:31:37,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:37,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48c37590270ad7b6d97c9 HTTP/1.1" 200 None 2026-03-26 01:31:37,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:37,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:31:37,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:37,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48c37590270ad7b6d97c9 HTTP/1.1" 200 None 2026-03-26 01:31:37,833 - lava_callback - INFO - Completed processing callback for node 69c48c37590270ad7b6d97c9 2026-03-26 01:31:40,468 - lava_callback - INFO - Received callback for node 69c1e96b0efaf21d93106626 INFO: 172.21.0.1:32972 - "POST /node/69c1e96b0efaf21d93106626 HTTP/1.0" 202 Accepted 2026-03-26 01:31:40,473 - lava_callback - INFO - Starting async job submit for node 69c1e96b0efaf21d93106626 2026-03-26 01:31:40,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:40,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e96b0efaf21d93106626 HTTP/1.1" 200 None 2026-03-26 01:31:40,500 - lava_callback - DEBUG - Processing node 69c1e96b0efaf21d93106626: baseline-arm64-clabbe 2026-03-26 01:31:40,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e96b0efaf21d93106626 2026-03-26 01:31:40,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:40,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e96b0efaf21d93106626 HTTP/1.1" 200 None 2026-03-26 01:31:40,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:40,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:31:40,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:31:40,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e96b0efaf21d93106626 HTTP/1.1" 200 None 2026-03-26 01:31:40,565 - lava_callback - ERROR - Error processing callback for node 69c1e96b0efaf21d93106626: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44986 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:32:00,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:32:00,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46422 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:33:00,576 - lava_callback - INFO - Received callback for node 69c1e9c30efaf21d9310665d INFO: 172.21.0.1:54870 - "POST /node/69c1e9c30efaf21d9310665d HTTP/1.0" 202 Accepted 2026-03-26 01:33:00,581 - lava_callback - INFO - Starting async job submit for node 69c1e9c30efaf21d9310665d 2026-03-26 01:33:00,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:33:00,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e9c30efaf21d9310665d HTTP/1.1" 200 None 2026-03-26 01:33:00,597 - lava_callback - DEBUG - Processing node 69c1e9c30efaf21d9310665d: baseline-arm64-clabbe 2026-03-26 01:33:00,597 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1e9c30efaf21d9310665d 2026-03-26 01:33:00,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:33:00,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e9c30efaf21d9310665d HTTP/1.1" 200 None 2026-03-26 01:33:00,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:33:00,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 01:33:00,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:33:00,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e9c30efaf21d9310665d HTTP/1.1" 200 None 2026-03-26 01:33:00,655 - lava_callback - ERROR - Error processing callback for node 69c1e9c30efaf21d9310665d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49950 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:34:11,533 - lava_callback - INFO - Received callback for node 69c4885e590270ad7b6d9559 INFO: 172.21.0.1:35612 - "POST /node/69c4885e590270ad7b6d9559 HTTP/1.0" 202 Accepted 2026-03-26 01:34:11,542 - lava_callback - INFO - Starting async job submit for node 69c4885e590270ad7b6d9559 2026-03-26 01:34:11,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:34:11,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885e590270ad7b6d9559 HTTP/1.1" 200 None 2026-03-26 01:34:11,601 - lava_callback - DEBUG - Processing node 69c4885e590270ad7b6d9559: baseline-nfs-arm64 2026-03-26 01:34:12,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:34:12,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:34:12,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885e590270ad7b6d9559/log.txt.gz 2026-03-26 01:34:13,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:34:13,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:34:13,886 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885e590270ad7b6d9559/lava-logs.yaml 2026-03-26 01:34:13,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:34:14,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:34:14,098 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4885e590270ad7b6d9559/lava_callback.json.gz DEBUG: Setup failed for 69c4885e590270ad7b6d9559. Transitting job node result: incomplete -> fail 2026-03-26 01:34:14,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:34:14,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4885e590270ad7b6d9559 HTTP/1.1" 200 None 2026-03-26 01:34:14,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:34:14,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:34:14,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:34:14,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4885e590270ad7b6d9559 HTTP/1.1" 200 None 2026-03-26 01:34:14,365 - lava_callback - INFO - Completed processing callback for node 69c4885e590270ad7b6d9559 INFO: 172.21.0.1:35624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58106 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:34:30,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:34:30,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39278 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 01:38:16,076 - lava_callback - INFO - Received callback for node 69c48c94590270ad7b6d97e4 INFO: 172.21.0.1:39286 - "POST /node/69c48c94590270ad7b6d97e4 HTTP/1.0" 202 Accepted 2026-03-26 01:38:16,086 - lava_callback - INFO - Starting async job submit for node 69c48c94590270ad7b6d97e4 2026-03-26 01:38:16,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:38:16,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48c94590270ad7b6d97e4 HTTP/1.1" 200 None 2026-03-26 01:38:16,157 - lava_callback - DEBUG - Processing node 69c48c94590270ad7b6d97e4: baseline-arm64-foundriesio INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:38:17,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:38:17,653 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:38:17,655 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c48c94590270ad7b6d97e4/log.txt.gz 2026-03-26 01:38:18,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:38:18,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:38:18,639 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c48c94590270ad7b6d97e4/lava-logs.yaml 2026-03-26 01:38:18,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:38:18,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:38:18,871 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c48c94590270ad7b6d97e4/lava_callback.json.gz DEBUG: Setup failed for 69c48c94590270ad7b6d97e4. Transitting job node result: incomplete -> fail 2026-03-26 01:38:18,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:38:18,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48c94590270ad7b6d97e4 HTTP/1.1" 200 None 2026-03-26 01:38:18,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:38:19,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:38:19,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:38:19,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48c94590270ad7b6d97e4 HTTP/1.1" 200 None 2026-03-26 01:38:19,155 - lava_callback - INFO - Completed processing callback for node 69c48c94590270ad7b6d97e4 INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:38:30,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:38:30,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:42:54,665 - lava_callback - INFO - Received callback for node 69c48f01590270ad7b6d9863 INFO: 172.21.0.1:51022 - "POST /node/69c48f01590270ad7b6d9863 HTTP/1.0" 202 Accepted 2026-03-26 01:42:54,671 - lava_callback - INFO - Starting async job submit for node 69c48f01590270ad7b6d9863 2026-03-26 01:42:54,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:42:54,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f01590270ad7b6d9863 HTTP/1.1" 200 None 2026-03-26 01:42:54,689 - lava_callback - DEBUG - Processing node 69c48f01590270ad7b6d9863: baseline-nfs-arm64 2026-03-26 01:42:54,691 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:42:54,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:42:54,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:42:54,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48f01590270ad7b6d9863/lava-logs.yaml 2026-03-26 01:42:54,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:42:54,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:42:54,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c48f01590270ad7b6d9863/lava_callback.json.gz 2026-03-26 01:42:54,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:42:55,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f01590270ad7b6d9863 HTTP/1.1" 200 None 2026-03-26 01:42:55,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:42:55,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:42:55,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:42:55,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48f01590270ad7b6d9863 HTTP/1.1" 200 None 2026-03-26 01:42:55,045 - lava_callback - INFO - Completed processing callback for node 69c48f01590270ad7b6d9863 INFO: 172.21.0.1:51038 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:43:00,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:43:00,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58084 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:43:58,519 - lava_callback - INFO - Received callback for node 69c48d8f590270ad7b6d9816 INFO: 172.21.0.1:58100 - "POST /node/69c48d8f590270ad7b6d9816 HTTP/1.0" 202 Accepted 2026-03-26 01:43:58,526 - lava_callback - INFO - Starting async job submit for node 69c48d8f590270ad7b6d9816 2026-03-26 01:43:58,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:43:58,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d8f590270ad7b6d9816 HTTP/1.1" 200 None 2026-03-26 01:43:58,577 - lava_callback - DEBUG - Processing node 69c48d8f590270ad7b6d9816: baseline-arm64 2026-03-26 01:43:58,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:43:59,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:43:59,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48d8f590270ad7b6d9816/log.txt.gz 2026-03-26 01:43:59,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:43:59,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:43:59,670 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48d8f590270ad7b6d9816/lava-logs.yaml 2026-03-26 01:43:59,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:43:59,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:43:59,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48d8f590270ad7b6d9816/lava_callback.json.gz DEBUG: Setup failed for 69c48d8f590270ad7b6d9816. Transitting job node result: incomplete -> fail 2026-03-26 01:43:59,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:43:59,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d8f590270ad7b6d9816 HTTP/1.1" 200 None 2026-03-26 01:43:59,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:43:59,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:43:59,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:43:59,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48d8f590270ad7b6d9816 HTTP/1.1" 200 None 2026-03-26 01:44:00,024 - lava_callback - INFO - Completed processing callback for node 69c48d8f590270ad7b6d9816 2026-03-26 01:44:00,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:44:00,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51586 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:45:12,844 - lava_callback - INFO - Received callback for node 69c1eca10efaf21d93106819 INFO: 172.21.0.1:43090 - "POST /node/69c1eca10efaf21d93106819 HTTP/1.0" 202 Accepted 2026-03-26 01:45:12,850 - lava_callback - INFO - Starting async job submit for node 69c1eca10efaf21d93106819 2026-03-26 01:45:12,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:45:12,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eca10efaf21d93106819 HTTP/1.1" 200 None 2026-03-26 01:45:12,867 - lava_callback - DEBUG - Processing node 69c1eca10efaf21d93106819: baseline-arm64-baylibre 2026-03-26 01:45:12,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1eca10efaf21d93106819 2026-03-26 01:45:12,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:45:12,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1eca10efaf21d93106819 HTTP/1.1" 200 None 2026-03-26 01:45:12,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:45:12,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:45:12,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:45:12,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1eca10efaf21d93106819 HTTP/1.1" 200 None 2026-03-26 01:45:12,926 - lava_callback - ERROR - Error processing callback for node 69c1eca10efaf21d93106819: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35652 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:46:05,216 - lava_callback - INFO - Received callback for node 69c48f9e590270ad7b6d988b INFO: 172.21.0.1:48622 - "POST /node/69c48f9e590270ad7b6d988b HTTP/1.0" 202 Accepted 2026-03-26 01:46:05,222 - lava_callback - INFO - Starting async job submit for node 69c48f9e590270ad7b6d988b 2026-03-26 01:46:05,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:05,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f9e590270ad7b6d988b HTTP/1.1" 200 None 2026-03-26 01:46:05,267 - lava_callback - DEBUG - Processing node 69c48f9e590270ad7b6d988b: baseline-arm64 2026-03-26 01:46:05,405 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:46:05,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:46:05,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:46:05,672 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48f9e590270ad7b6d988b/lava-logs.yaml 2026-03-26 01:46:05,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:46:05,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:46:05,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c48f9e590270ad7b6d988b/lava_callback.json.gz 2026-03-26 01:46:05,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:05,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f9e590270ad7b6d988b HTTP/1.1" 200 None 2026-03-26 01:46:05,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:05,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:46:05,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:05,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48f9e590270ad7b6d988b HTTP/1.1" 200 None 2026-03-26 01:46:05,996 - lava_callback - INFO - Completed processing callback for node 69c48f9e590270ad7b6d988b INFO: 172.21.0.1:48628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54348 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:46:30,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:30,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:46:52,850 - lava_callback - INFO - Received callback for node 69c1ecfa0efaf21d9310684e INFO: 172.21.0.1:51590 - "POST /node/69c1ecfa0efaf21d9310684e HTTP/1.0" 202 Accepted 2026-03-26 01:46:52,855 - lava_callback - INFO - Starting async job submit for node 69c1ecfa0efaf21d9310684e 2026-03-26 01:46:52,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:52,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ecfa0efaf21d9310684e HTTP/1.1" 200 None 2026-03-26 01:46:52,873 - lava_callback - DEBUG - Processing node 69c1ecfa0efaf21d9310684e: baseline-arm64-baylibre 2026-03-26 01:46:52,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1ecfa0efaf21d9310684e 2026-03-26 01:46:52,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:52,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ecfa0efaf21d9310684e HTTP/1.1" 200 None 2026-03-26 01:46:52,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:52,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:46:52,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:46:52,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ecfa0efaf21d9310684e HTTP/1.1" 200 None 2026-03-26 01:46:52,932 - lava_callback - ERROR - Error processing callback for node 69c1ecfa0efaf21d9310684e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51598 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:47:01,251 - lava_callback - INFO - Received callback for node 69c48be3590270ad7b6d97b0 INFO: 172.21.0.1:54068 - "POST /node/69c48be3590270ad7b6d97b0 HTTP/1.0" 202 Accepted 2026-03-26 01:47:01,258 - lava_callback - INFO - Starting async job submit for node 69c48be3590270ad7b6d97b0 2026-03-26 01:47:01,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:47:01,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48be3590270ad7b6d97b0 HTTP/1.1" 200 None 2026-03-26 01:47:01,331 - lava_callback - DEBUG - Processing node 69c48be3590270ad7b6d97b0: ltp-smoke 2026-03-26 01:47:01,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:47:01,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:47:01,975 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c48be3590270ad7b6d97b0/log.txt.gz 2026-03-26 01:47:02,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:47:02,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:47:02,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c48be3590270ad7b6d97b0/lava-logs.yaml 2026-03-26 01:47:02,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:47:02,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:47:02,711 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c48be3590270ad7b6d97b0/lava_callback.json.gz 2026-03-26 01:47:02,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:47:02,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48be3590270ad7b6d97b0 HTTP/1.1" 200 None 2026-03-26 01:47:02,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:47:02,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b41590270ad7b6d6994 HTTP/1.1" 200 None 2026-03-26 01:47:02,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:47:02,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48be3590270ad7b6d97b0 HTTP/1.1" 200 None 2026-03-26 01:47:03,011 - lava_callback - INFO - Completed processing callback for node 69c48be3590270ad7b6d97b0 INFO: 172.21.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56224 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:47:30,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:47:30,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60246 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:48:12,902 - lava_callback - INFO - Received callback for node 69c1ed4c0efaf21d93106881 INFO: 172.21.0.1:49290 - "POST /node/69c1ed4c0efaf21d93106881 HTTP/1.0" 202 Accepted 2026-03-26 01:48:12,907 - lava_callback - INFO - Starting async job submit for node 69c1ed4c0efaf21d93106881 2026-03-26 01:48:12,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:48:12,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ed4c0efaf21d93106881 HTTP/1.1" 200 None 2026-03-26 01:48:12,923 - lava_callback - DEBUG - Processing node 69c1ed4c0efaf21d93106881: baseline-arm64-baylibre 2026-03-26 01:48:12,923 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1ed4c0efaf21d93106881 2026-03-26 01:48:12,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:48:12,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ed4c0efaf21d93106881 HTTP/1.1" 200 None 2026-03-26 01:48:12,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:48:12,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:48:12,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:48:12,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ed4c0efaf21d93106881 HTTP/1.1" 200 None 2026-03-26 01:48:12,986 - lava_callback - ERROR - Error processing callback for node 69c1ed4c0efaf21d93106881: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39152 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:49:07,114 - lava_callback - INFO - Received callback for node 69c49040590270ad7b6d98b2 INFO: 172.21.0.1:39160 - "POST /node/69c49040590270ad7b6d98b2 HTTP/1.0" 202 Accepted 2026-03-26 01:49:07,120 - lava_callback - INFO - Starting async job submit for node 69c49040590270ad7b6d98b2 2026-03-26 01:49:07,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:07,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49040590270ad7b6d98b2 HTTP/1.1" 200 None 2026-03-26 01:49:07,170 - lava_callback - DEBUG - Processing node 69c49040590270ad7b6d98b2: baseline-arm64 2026-03-26 01:49:07,304 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:49:07,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:49:07,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:49:07,572 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c49040590270ad7b6d98b2/lava-logs.yaml 2026-03-26 01:49:07,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:49:07,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:49:07,870 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c49040590270ad7b6d98b2/lava_callback.json.gz 2026-03-26 01:49:07,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:07,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49040590270ad7b6d98b2 HTTP/1.1" 200 None 2026-03-26 01:49:07,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:07,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:49:07,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:07,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49040590270ad7b6d98b2 HTTP/1.1" 200 None 2026-03-26 01:49:08,036 - lava_callback - INFO - Completed processing callback for node 69c49040590270ad7b6d98b2 INFO: 172.21.0.1:40548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54918 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:49:30,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:30,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50820 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:49:41,363 - lava_callback - INFO - Received callback for node 69c1edaf0efaf21d931068ab INFO: 172.21.0.1:43124 - "POST /node/69c1edaf0efaf21d931068ab HTTP/1.0" 202 Accepted 2026-03-26 01:49:41,368 - lava_callback - INFO - Starting async job submit for node 69c1edaf0efaf21d931068ab 2026-03-26 01:49:41,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:41,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1edaf0efaf21d931068ab HTTP/1.1" 200 None 2026-03-26 01:49:41,384 - lava_callback - DEBUG - Processing node 69c1edaf0efaf21d931068ab: baseline-arm64-clabbe 2026-03-26 01:49:41,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1edaf0efaf21d931068ab 2026-03-26 01:49:41,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:41,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1edaf0efaf21d931068ab HTTP/1.1" 200 None 2026-03-26 01:49:41,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:41,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:49:41,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:49:41,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1edaf0efaf21d931068ab HTTP/1.1" 200 None 2026-03-26 01:49:41,443 - lava_callback - ERROR - Error processing callback for node 69c1edaf0efaf21d931068ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39016 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:51:21,343 - lava_callback - INFO - Received callback for node 69c1ee070efaf21d931068d2 INFO: 172.21.0.1:58406 - "POST /node/69c1ee070efaf21d931068d2 HTTP/1.0" 202 Accepted 2026-03-26 01:51:21,349 - lava_callback - INFO - Starting async job submit for node 69c1ee070efaf21d931068d2 2026-03-26 01:51:21,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:51:21,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ee070efaf21d931068d2 HTTP/1.1" 200 None 2026-03-26 01:51:21,366 - lava_callback - DEBUG - Processing node 69c1ee070efaf21d931068d2: baseline-arm64-clabbe 2026-03-26 01:51:21,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1ee070efaf21d931068d2 2026-03-26 01:51:21,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:51:21,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ee070efaf21d931068d2 HTTP/1.1" 200 None 2026-03-26 01:51:21,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:51:21,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:51:21,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:51:21,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ee070efaf21d931068d2 HTTP/1.1" 200 None 2026-03-26 01:51:21,426 - lava_callback - ERROR - Error processing callback for node 69c1ee070efaf21d931068d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40324 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:52:14,741 - lava_callback - INFO - Received callback for node 69c48d16590270ad7b6d97fd INFO: 172.21.0.1:54268 - "POST /node/69c48d16590270ad7b6d97fd HTTP/1.0" 202 Accepted 2026-03-26 01:52:14,748 - lava_callback - INFO - Starting async job submit for node 69c48d16590270ad7b6d97fd 2026-03-26 01:52:14,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:52:14,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d16590270ad7b6d97fd HTTP/1.1" 200 None 2026-03-26 01:52:14,816 - lava_callback - DEBUG - Processing node 69c48d16590270ad7b6d97fd: baseline-arm INFO: 172.21.0.1:54274 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 01:52:15,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:52:15,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:52:15,548 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c48d16590270ad7b6d97fd/log.txt.gz 2026-03-26 01:52:15,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:52:16,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:52:16,169 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c48d16590270ad7b6d97fd/lava-logs.yaml 2026-03-26 01:52:16,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:52:16,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:52:16,379 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c48d16590270ad7b6d97fd/lava_callback.json.gz DEBUG: Setup failed for 69c48d16590270ad7b6d97fd. Transitting job node result: incomplete -> fail 2026-03-26 01:52:16,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:52:16,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d16590270ad7b6d97fd HTTP/1.1" 200 None 2026-03-26 01:52:16,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:54286 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:52:16,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 01:52:16,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:52:16,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48d16590270ad7b6d97fd HTTP/1.1" 200 None 2026-03-26 01:52:16,650 - lava_callback - INFO - Completed processing callback for node 69c48d16590270ad7b6d97fd INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:52:30,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:52:30,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47292 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:53:51,800 - lava_callback - INFO - Received callback for node 69c4887a590270ad7b6d95b4 INFO: 172.21.0.1:47724 - "POST /node/69c4887a590270ad7b6d95b4 HTTP/1.0" 202 Accepted 2026-03-26 01:53:51,807 - lava_callback - INFO - Starting async job submit for node 69c4887a590270ad7b6d95b4 2026-03-26 01:53:51,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:53:51,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887a590270ad7b6d95b4 HTTP/1.1" 200 None 2026-03-26 01:53:51,844 - lava_callback - DEBUG - Processing node 69c4887a590270ad7b6d95b4: baseline-arm64-broonie 2026-03-26 01:53:52,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:53:52,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:53:52,285 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887a590270ad7b6d95b4/log.txt.gz 2026-03-26 01:53:52,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:53:52,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:53:52,675 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887a590270ad7b6d95b4/lava-logs.yaml 2026-03-26 01:53:52,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:47732 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:53:52,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:53:52,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4887a590270ad7b6d95b4/lava_callback.json.gz 2026-03-26 01:53:52,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:53:52,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887a590270ad7b6d95b4 HTTP/1.1" 200 None 2026-03-26 01:53:52,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:53:52,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:53:52,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:53:52,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4887a590270ad7b6d95b4 HTTP/1.1" 200 None 2026-03-26 01:53:52,951 - lava_callback - INFO - Completed processing callback for node 69c4887a590270ad7b6d95b4 INFO: 172.21.0.1:47748 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:54:00,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:54:00,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44228 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:55:41,473 - lava_callback - INFO - Received callback for node 69c1ef0d0efaf21d93106953 INFO: 172.21.0.1:58516 - "POST /node/69c1ef0d0efaf21d93106953 HTTP/1.0" 202 Accepted 2026-03-26 01:55:41,479 - lava_callback - INFO - Starting async job submit for node 69c1ef0d0efaf21d93106953 2026-03-26 01:55:41,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:55:41,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ef0d0efaf21d93106953 HTTP/1.1" 200 None 2026-03-26 01:55:41,494 - lava_callback - DEBUG - Processing node 69c1ef0d0efaf21d93106953: baseline-arm64-clabbe 2026-03-26 01:55:41,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1ef0d0efaf21d93106953 2026-03-26 01:55:41,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:55:41,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ef0d0efaf21d93106953 HTTP/1.1" 200 None 2026-03-26 01:55:41,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:55:41,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:55:41,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:55:41,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ef0d0efaf21d93106953 HTTP/1.1" 200 None 2026-03-26 01:55:41,572 - lava_callback - ERROR - Error processing callback for node 69c1ef0d0efaf21d93106953: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:56:01,403 - lava_callback - INFO - Received callback for node 69c48878590270ad7b6d95ad INFO: 172.21.0.1:38580 - "POST /node/69c48878590270ad7b6d95ad HTTP/1.0" 202 Accepted 2026-03-26 01:56:01,409 - lava_callback - INFO - Starting async job submit for node 69c48878590270ad7b6d95ad 2026-03-26 01:56:01,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:01,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48878590270ad7b6d95ad HTTP/1.1" 200 None 2026-03-26 01:56:01,459 - lava_callback - DEBUG - Processing node 69c48878590270ad7b6d95ad: baseline-arm64-broonie 2026-03-26 01:56:01,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:56:02,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:56:02,012 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48878590270ad7b6d95ad/log.txt.gz 2026-03-26 01:56:02,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:56:02,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:56:02,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48878590270ad7b6d95ad/lava-logs.yaml 2026-03-26 01:56:02,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:56:02,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:56:02,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48878590270ad7b6d95ad/lava_callback.json.gz 2026-03-26 01:56:02,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:02,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48878590270ad7b6d95ad HTTP/1.1" 200 None 2026-03-26 01:56:02,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:02,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:56:02,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:02,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48878590270ad7b6d95ad HTTP/1.1" 200 None 2026-03-26 01:56:02,650 - lava_callback - INFO - Completed processing callback for node 69c48878590270ad7b6d95ad INFO: 172.21.0.1:38596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49848 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:56:30,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:30,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56804 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:56:53,521 - lava_callback - INFO - Received callback for node 69c1ef560efaf21d93106978 INFO: 172.21.0.1:56816 - "POST /node/69c1ef560efaf21d93106978 HTTP/1.0" 202 Accepted 2026-03-26 01:56:53,526 - lava_callback - INFO - Starting async job submit for node 69c1ef560efaf21d93106978 2026-03-26 01:56:53,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:53,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ef560efaf21d93106978 HTTP/1.1" 200 None 2026-03-26 01:56:53,543 - lava_callback - DEBUG - Processing node 69c1ef560efaf21d93106978: baseline-arm64-baylibre 2026-03-26 01:56:53,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1ef560efaf21d93106978 2026-03-26 01:56:53,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:53,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ef560efaf21d93106978 HTTP/1.1" 200 None 2026-03-26 01:56:53,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:53,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:56:53,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:56:53,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ef560efaf21d93106978 HTTP/1.1" 200 None 2026-03-26 01:56:53,616 - lava_callback - ERROR - Error processing callback for node 69c1ef560efaf21d93106978: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40758 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:57:35,274 - lava_callback - INFO - Received callback for node 69c49277590270ad7b6d9928 INFO: 172.21.0.1:54366 - "POST /node/69c49277590270ad7b6d9928 HTTP/1.0" 202 Accepted 2026-03-26 01:57:35,280 - lava_callback - INFO - Starting async job submit for node 69c49277590270ad7b6d9928 2026-03-26 01:57:35,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:57:35,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49277590270ad7b6d9928 HTTP/1.1" 200 None 2026-03-26 01:57:35,304 - lava_callback - DEBUG - Processing node 69c49277590270ad7b6d9928: baseline-nfs-arm64 2026-03-26 01:57:35,305 - lava_callback - WARNING - Failed to upload log 2026-03-26 01:57:35,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:57:35,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:57:35,626 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49277590270ad7b6d9928/lava-logs.yaml 2026-03-26 01:57:35,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 01:57:35,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 01:57:35,778 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49277590270ad7b6d9928/lava_callback.json.gz 2026-03-26 01:57:35,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:57:35,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49277590270ad7b6d9928 HTTP/1.1" 200 None 2026-03-26 01:57:35,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:57:35,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 01:57:35,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:57:35,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49277590270ad7b6d9928 HTTP/1.1" 200 None 2026-03-26 01:57:35,853 - lava_callback - INFO - Completed processing callback for node 69c49277590270ad7b6d9928 INFO: 172.21.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54972 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:58:00,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:58:00,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK 2026-03-26 01:58:21,577 - lava_callback - INFO - Received callback for node 69c1efb80efaf21d931069ab INFO: 172.21.0.1:57956 - "POST /node/69c1efb80efaf21d931069ab HTTP/1.0" 202 Accepted 2026-03-26 01:58:21,582 - lava_callback - INFO - Starting async job submit for node 69c1efb80efaf21d931069ab 2026-03-26 01:58:21,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:58:21,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1efb80efaf21d931069ab HTTP/1.1" 200 None 2026-03-26 01:58:21,598 - lava_callback - DEBUG - Processing node 69c1efb80efaf21d931069ab: baseline-arm64-clabbe 2026-03-26 01:58:21,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1efb80efaf21d931069ab 2026-03-26 01:58:21,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:58:21,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1efb80efaf21d931069ab HTTP/1.1" 200 None 2026-03-26 01:58:21,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:58:21,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 01:58:21,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 01:58:21,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1efb80efaf21d931069ab HTTP/1.1" 200 None 2026-03-26 01:58:21,656 - lava_callback - ERROR - Error processing callback for node 69c1efb80efaf21d931069ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57768 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:00:01,690 - lava_callback - INFO - Received callback for node 69c1f0120efaf21d931069c8 INFO: 172.21.0.1:55250 - "POST /node/69c1f0120efaf21d931069c8 HTTP/1.0" 202 Accepted 2026-03-26 02:00:01,696 - lava_callback - INFO - Starting async job submit for node 69c1f0120efaf21d931069c8 2026-03-26 02:00:01,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:01,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f0120efaf21d931069c8 HTTP/1.1" 200 None 2026-03-26 02:00:01,712 - lava_callback - DEBUG - Processing node 69c1f0120efaf21d931069c8: baseline-arm64-clabbe 2026-03-26 02:00:01,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f0120efaf21d931069c8 2026-03-26 02:00:01,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:01,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f0120efaf21d931069c8 HTTP/1.1" 200 None 2026-03-26 02:00:01,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:01,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:00:01,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:01,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f0120efaf21d931069c8 HTTP/1.1" 200 None 2026-03-26 02:00:01,786 - lava_callback - ERROR - Error processing callback for node 69c1f0120efaf21d931069c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55266 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:00:06,938 - lava_callback - INFO - Received callback for node 69c491e8590270ad7b6d990b INFO: 172.21.0.1:55274 - "POST /node/69c491e8590270ad7b6d990b HTTP/1.0" 202 Accepted 2026-03-26 02:00:06,947 - lava_callback - INFO - Starting async job submit for node 69c491e8590270ad7b6d990b 2026-03-26 02:00:06,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:07,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c491e8590270ad7b6d990b HTTP/1.1" 200 None 2026-03-26 02:00:07,009 - lava_callback - DEBUG - Processing node 69c491e8590270ad7b6d990b: baseline-arm64-foundriesio 2026-03-26 02:00:08,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:00:08,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:00:08,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c491e8590270ad7b6d990b/log.txt.gz 2026-03-26 02:00:09,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:00:09,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:00:09,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c491e8590270ad7b6d990b/lava-logs.yaml 2026-03-26 02:00:09,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:00:09,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:00:09,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c491e8590270ad7b6d990b/lava_callback.json.gz DEBUG: 69c491e8590270ad7b6d990b Transitting job node result: pass -> incomplete 2026-03-26 02:00:09,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:09,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c491e8590270ad7b6d990b HTTP/1.1" 200 None 2026-03-26 02:00:09,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:09,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:00:09,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:09,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c491e8590270ad7b6d990b HTTP/1.1" 200 None 2026-03-26 02:00:09,588 - lava_callback - INFO - Completed processing callback for node 69c491e8590270ad7b6d990b INFO: 172.21.0.1:42600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:00:30,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:00:30,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33866 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:01:06,712 - lava_callback - INFO - Received callback for node 69c4930f590270ad7b6d9940 INFO: 172.21.0.1:33880 - "POST /node/69c4930f590270ad7b6d9940 HTTP/1.0" 202 Accepted 2026-03-26 02:01:06,735 - lava_callback - INFO - Starting async job submit for node 69c4930f590270ad7b6d9940 2026-03-26 02:01:06,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:06,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4930f590270ad7b6d9940 HTTP/1.1" 200 None 2026-03-26 02:01:06,782 - lava_callback - DEBUG - Processing node 69c4930f590270ad7b6d9940: baseline-arm64 2026-03-26 02:01:06,914 - lava_callback - WARNING - Failed to upload log 2026-03-26 02:01:06,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:01:07,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:01:07,156 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4930f590270ad7b6d9940/lava-logs.yaml 2026-03-26 02:01:07,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:01:07,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:01:07,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4930f590270ad7b6d9940/lava_callback.json.gz 2026-03-26 02:01:07,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:07,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4930f590270ad7b6d9940 HTTP/1.1" 200 None 2026-03-26 02:01:07,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:07,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:01:07,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:07,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4930f590270ad7b6d9940 HTTP/1.1" 200 None 2026-03-26 02:01:07,420 - lava_callback - INFO - Completed processing callback for node 69c4930f590270ad7b6d9940 2026-03-26 02:01:13,619 - lava_callback - INFO - Received callback for node 69c1f0670efaf21d931069ef INFO: 172.21.0.1:40138 - "POST /node/69c1f0670efaf21d931069ef HTTP/1.0" 202 Accepted 2026-03-26 02:01:13,625 - lava_callback - INFO - Starting async job submit for node 69c1f0670efaf21d931069ef 2026-03-26 02:01:13,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:13,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f0670efaf21d931069ef HTTP/1.1" 200 None 2026-03-26 02:01:13,663 - lava_callback - DEBUG - Processing node 69c1f0670efaf21d931069ef: baseline-arm64-baylibre 2026-03-26 02:01:13,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f0670efaf21d931069ef 2026-03-26 02:01:13,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:13,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f0670efaf21d931069ef HTTP/1.1" 200 None 2026-03-26 02:01:13,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:13,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:01:13,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:13,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f0670efaf21d931069ef HTTP/1.1" 200 None 2026-03-26 02:01:13,749 - lava_callback - ERROR - Error processing callback for node 69c1f0670efaf21d931069ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:01:25,063 - lava_callback - INFO - Received callback for node 69c48edf590270ad7b6d9858 INFO: 172.21.0.1:40844 - "POST /node/69c48edf590270ad7b6d9858 HTTP/1.0" 202 Accepted 2026-03-26 02:01:25,109 - lava_callback - INFO - Starting async job submit for node 69c48edf590270ad7b6d9858 2026-03-26 02:01:25,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:25,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48edf590270ad7b6d9858 HTTP/1.1" 200 None 2026-03-26 02:01:25,150 - lava_callback - DEBUG - Processing node 69c48edf590270ad7b6d9858: baseline-arm-pengutronix 2026-03-26 02:01:25,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:01:25,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:01:25,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48edf590270ad7b6d9858/log.txt.gz 2026-03-26 02:01:25,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:01:25,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:01:25,661 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48edf590270ad7b6d9858/lava-logs.yaml 2026-03-26 02:01:25,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:01:25,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:01:25,777 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c48edf590270ad7b6d9858/lava_callback.json.gz DEBUG: Setup failed for 69c48edf590270ad7b6d9858. Transitting job node result: incomplete -> fail 2026-03-26 02:01:25,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:25,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48edf590270ad7b6d9858 HTTP/1.1" 200 None 2026-03-26 02:01:25,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:25,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 02:01:25,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:25,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48edf590270ad7b6d9858 HTTP/1.1" 200 None 2026-03-26 02:01:25,954 - lava_callback - INFO - Completed processing callback for node 69c48edf590270ad7b6d9858 INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:01:30,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:01:30,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47864 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:02:05,749 - lava_callback - INFO - Received callback for node 69c49361590270ad7b6d9958 INFO: 172.21.0.1:58486 - "POST /node/69c49361590270ad7b6d9958 HTTP/1.0" 202 Accepted 2026-03-26 02:02:05,756 - lava_callback - INFO - Starting async job submit for node 69c49361590270ad7b6d9958 2026-03-26 02:02:05,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:02:05,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49361590270ad7b6d9958 HTTP/1.1" 200 None 2026-03-26 02:02:05,800 - lava_callback - DEBUG - Processing node 69c49361590270ad7b6d9958: baseline-arm64 2026-03-26 02:02:05,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:02:06,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:02:06,077 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c49361590270ad7b6d9958/log.txt.gz 2026-03-26 02:02:06,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:02:06,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:02:06,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c49361590270ad7b6d9958/lava-logs.yaml 2026-03-26 02:02:06,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:02:06,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:02:06,480 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c49361590270ad7b6d9958/lava_callback.json.gz 2026-03-26 02:02:06,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:02:06,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49361590270ad7b6d9958 HTTP/1.1" 200 None 2026-03-26 02:02:06,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:58492 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:02:06,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:02:06,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:02:06,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49361590270ad7b6d9958 HTTP/1.1" 200 None 2026-03-26 02:02:06,656 - lava_callback - INFO - Completed processing callback for node 69c49361590270ad7b6d9958 INFO: 172.21.0.1:44734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58956 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:02:30,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:02:30,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42050 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:03:01,829 - lava_callback - INFO - Received callback for node 69c1f0c50efaf21d93106a1c INFO: 172.21.0.1:44156 - "POST /node/69c1f0c50efaf21d93106a1c HTTP/1.0" 202 Accepted 2026-03-26 02:03:01,834 - lava_callback - INFO - Starting async job submit for node 69c1f0c50efaf21d93106a1c 2026-03-26 02:03:01,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:03:01,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f0c50efaf21d93106a1c HTTP/1.1" 200 None 2026-03-26 02:03:01,850 - lava_callback - DEBUG - Processing node 69c1f0c50efaf21d93106a1c: baseline-arm64-clabbe 2026-03-26 02:03:01,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f0c50efaf21d93106a1c 2026-03-26 02:03:01,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:03:01,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f0c50efaf21d93106a1c HTTP/1.1" 200 None 2026-03-26 02:03:01,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:03:01,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:03:01,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:03:01,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f0c50efaf21d93106a1c HTTP/1.1" 200 None 2026-03-26 02:03:01,920 - lava_callback - ERROR - Error processing callback for node 69c1f0c50efaf21d93106a1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39452 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:04:13,837 - lava_callback - INFO - Received callback for node 69c1f1100efaf21d93106a3d INFO: 172.21.0.1:32812 - "POST /node/69c1f1100efaf21d93106a3d HTTP/1.0" 202 Accepted 2026-03-26 02:04:13,842 - lava_callback - INFO - Starting async job submit for node 69c1f1100efaf21d93106a3d 2026-03-26 02:04:13,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:04:13,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f1100efaf21d93106a3d HTTP/1.1" 200 None 2026-03-26 02:04:13,858 - lava_callback - DEBUG - Processing node 69c1f1100efaf21d93106a3d: baseline-arm64-baylibre 2026-03-26 02:04:13,858 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f1100efaf21d93106a3d 2026-03-26 02:04:13,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:04:13,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f1100efaf21d93106a3d HTTP/1.1" 200 None 2026-03-26 02:04:13,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:04:13,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:04:13,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:04:13,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f1100efaf21d93106a3d HTTP/1.1" 200 None 2026-03-26 02:04:13,982 - lava_callback - ERROR - Error processing callback for node 69c1f1100efaf21d93106a3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50382 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:05:54,639 - lava_callback - INFO - Received callback for node 69c492c5590270ad7b6d9935 INFO: 172.21.0.1:50384 - "POST /node/69c492c5590270ad7b6d9935 HTTP/1.0" 202 Accepted 2026-03-26 02:05:54,646 - lava_callback - INFO - Starting async job submit for node 69c492c5590270ad7b6d9935 2026-03-26 02:05:54,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:05:54,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c492c5590270ad7b6d9935 HTTP/1.1" 200 None 2026-03-26 02:05:54,696 - lava_callback - DEBUG - Processing node 69c492c5590270ad7b6d9935: baseline-arm64 2026-03-26 02:05:55,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:05:55,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:05:55,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c492c5590270ad7b6d9935/log.txt.gz 2026-03-26 02:05:55,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:05:55,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:05:55,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c492c5590270ad7b6d9935/lava-logs.yaml 2026-03-26 02:05:55,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:05:56,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:05:56,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c492c5590270ad7b6d9935/lava_callback.json.gz DEBUG: Setup failed for 69c492c5590270ad7b6d9935. Transitting job node result: incomplete -> fail 2026-03-26 02:05:56,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:05:56,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c492c5590270ad7b6d9935 HTTP/1.1" 200 None 2026-03-26 02:05:56,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:05:56,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:05:56,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:05:56,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c492c5590270ad7b6d9935 HTTP/1.1" 200 None 2026-03-26 02:05:56,394 - lava_callback - INFO - Completed processing callback for node 69c492c5590270ad7b6d9935 INFO: 172.21.0.1:50386 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:06:00,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:06:00,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46346 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:07:21,975 - lava_callback - INFO - Received callback for node 69c1f1c60efaf21d93106a87 INFO: 172.21.0.1:41244 - "POST /node/69c1f1c60efaf21d93106a87 HTTP/1.0" 202 Accepted 2026-03-26 02:07:21,981 - lava_callback - INFO - Starting async job submit for node 69c1f1c60efaf21d93106a87 2026-03-26 02:07:21,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:07:21,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f1c60efaf21d93106a87 HTTP/1.1" 200 None 2026-03-26 02:07:21,997 - lava_callback - DEBUG - Processing node 69c1f1c60efaf21d93106a87: baseline-arm64-clabbe 2026-03-26 02:07:21,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f1c60efaf21d93106a87 2026-03-26 02:07:21,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:07:22,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f1c60efaf21d93106a87 HTTP/1.1" 200 None 2026-03-26 02:07:22,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:07:22,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:07:22,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:07:22,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f1c60efaf21d93106a87 HTTP/1.1" 200 None 2026-03-26 02:07:22,056 - lava_callback - ERROR - Error processing callback for node 69c1f1c60efaf21d93106a87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55122 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:08:14,461 - lava_callback - INFO - Received callback for node 69c493ee590270ad7b6d9979 INFO: 172.21.0.1:55836 - "POST /node/69c493ee590270ad7b6d9979 HTTP/1.0" 202 Accepted 2026-03-26 02:08:14,471 - lava_callback - INFO - Starting async job submit for node 69c493ee590270ad7b6d9979 2026-03-26 02:08:14,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:08:14,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c493ee590270ad7b6d9979 HTTP/1.1" 200 None 2026-03-26 02:08:14,554 - lava_callback - DEBUG - Processing node 69c493ee590270ad7b6d9979: baseline-arm64-foundriesio INFO: 172.21.0.1:55848 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 02:08:15,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:08:15,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:08:15,922 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c493ee590270ad7b6d9979/log.txt.gz INFO: 172.21.0.1:55862 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:08:16,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:08:16,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:08:16,852 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c493ee590270ad7b6d9979/lava-logs.yaml 2026-03-26 02:08:16,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:08:17,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:08:17,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c493ee590270ad7b6d9979/lava_callback.json.gz DEBUG: 69c493ee590270ad7b6d9979 Transitting job node result: pass -> incomplete 2026-03-26 02:08:17,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:08:17,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c493ee590270ad7b6d9979 HTTP/1.1" 200 None 2026-03-26 02:08:17,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:08:17,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:08:17,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:08:17,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c493ee590270ad7b6d9979 HTTP/1.1" 200 None 2026-03-26 02:08:17,229 - lava_callback - INFO - Completed processing callback for node 69c493ee590270ad7b6d9979 INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:08:30,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:08:30,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32886 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:10:14,088 - lava_callback - INFO - Received callback for node 69c1f2720efaf21d93106ad3 INFO: 172.21.0.1:48472 - "POST /node/69c1f2720efaf21d93106ad3 HTTP/1.0" 202 Accepted 2026-03-26 02:10:14,094 - lava_callback - INFO - Starting async job submit for node 69c1f2720efaf21d93106ad3 2026-03-26 02:10:14,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:10:14,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f2720efaf21d93106ad3 HTTP/1.1" 200 None 2026-03-26 02:10:14,121 - lava_callback - DEBUG - Processing node 69c1f2720efaf21d93106ad3: baseline-arm64-baylibre 2026-03-26 02:10:14,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f2720efaf21d93106ad3 2026-03-26 02:10:14,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:10:14,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f2720efaf21d93106ad3 HTTP/1.1" 200 None 2026-03-26 02:10:14,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:10:14,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:10:14,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:10:14,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f2720efaf21d93106ad3 HTTP/1.1" 200 None 2026-03-26 02:10:14,184 - lava_callback - ERROR - Error processing callback for node 69c1f2720efaf21d93106ad3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:11:30,831 - lava_callback - INFO - Received callback for node 69c4948b590270ad7b6d9996 INFO: 172.21.0.1:50964 - "POST /node/69c4948b590270ad7b6d9996 HTTP/1.0" 202 Accepted 2026-03-26 02:11:30,838 - lava_callback - INFO - Starting async job submit for node 69c4948b590270ad7b6d9996 2026-03-26 02:11:30,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:11:30,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4948b590270ad7b6d9996 HTTP/1.1" 200 None 2026-03-26 02:11:30,901 - lava_callback - DEBUG - Processing node 69c4948b590270ad7b6d9996: baseline-arm64 2026-03-26 02:11:31,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:11:31,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:11:31,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4948b590270ad7b6d9996/log.txt.gz 2026-03-26 02:11:31,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:11:31,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:11:31,924 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4948b590270ad7b6d9996/lava-logs.yaml 2026-03-26 02:11:31,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:11:32,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:11:32,121 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c4948b590270ad7b6d9996/lava_callback.json.gz DEBUG: Setup failed for 69c4948b590270ad7b6d9996. Transitting job node result: incomplete -> fail 2026-03-26 02:11:32,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:11:32,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4948b590270ad7b6d9996 HTTP/1.1" 200 None 2026-03-26 02:11:32,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:11:32,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:11:32,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:11:32,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4948b590270ad7b6d9996 HTTP/1.1" 200 None 2026-03-26 02:11:32,331 - lava_callback - INFO - Completed processing callback for node 69c4948b590270ad7b6d9996 INFO: 172.21.0.1:50968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49402 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:12:00,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:12:00,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54746 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:13:02,303 - lava_callback - INFO - Received callback for node 69c1f3250efaf21d93106b30 INFO: 172.21.0.1:36682 - "POST /node/69c1f3250efaf21d93106b30 HTTP/1.0" 202 Accepted 2026-03-26 02:13:02,309 - lava_callback - INFO - Starting async job submit for node 69c1f3250efaf21d93106b30 2026-03-26 02:13:02,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:02,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f3250efaf21d93106b30 HTTP/1.1" 200 None 2026-03-26 02:13:02,325 - lava_callback - DEBUG - Processing node 69c1f3250efaf21d93106b30: baseline-arm64-clabbe 2026-03-26 02:13:02,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f3250efaf21d93106b30 2026-03-26 02:13:02,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:02,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f3250efaf21d93106b30 HTTP/1.1" 200 None 2026-03-26 02:13:02,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:02,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:13:02,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:02,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f3250efaf21d93106b30 HTTP/1.1" 200 None 2026-03-26 02:13:02,387 - lava_callback - ERROR - Error processing callback for node 69c1f3250efaf21d93106b30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:13:12,931 - lava_callback - INFO - Received callback for node 69c4918c590270ad7b6d98f5 INFO: 172.21.0.1:40610 - "POST /node/69c4918c590270ad7b6d98f5 HTTP/1.0" 202 Accepted 2026-03-26 02:13:12,938 - lava_callback - INFO - Starting async job submit for node 69c4918c590270ad7b6d98f5 2026-03-26 02:13:12,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:12,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4918c590270ad7b6d98f5 HTTP/1.1" 200 None 2026-03-26 02:13:12,997 - lava_callback - DEBUG - Processing node 69c4918c590270ad7b6d98f5: baseline-nfs-arm64 2026-03-26 02:13:14,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:13:14,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:13:14,260 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4918c590270ad7b6d98f5/log.txt.gz 2026-03-26 02:13:14,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:13:15,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:13:15,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4918c590270ad7b6d98f5/lava-logs.yaml INFO: 172.21.0.1:40618 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 02:13:15,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:13:15,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:13:15,357 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c4918c590270ad7b6d98f5/lava_callback.json.gz DEBUG: Setup failed for 69c4918c590270ad7b6d98f5. Transitting job node result: incomplete -> fail 2026-03-26 02:13:15,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:15,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4918c590270ad7b6d98f5 HTTP/1.1" 200 None 2026-03-26 02:13:15,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:15,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:13:15,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:15,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4918c590270ad7b6d98f5 HTTP/1.1" 200 None 2026-03-26 02:13:15,599 - lava_callback - INFO - Completed processing callback for node 69c4918c590270ad7b6d98f5 INFO: 172.21.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48338 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:13:30,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:13:30,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57478 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:16:02,445 - lava_callback - INFO - Received callback for node 69c1f3d10efaf21d93106b79 INFO: 172.21.0.1:40646 - "POST /node/69c1f3d10efaf21d93106b79 HTTP/1.0" 202 Accepted 2026-03-26 02:16:02,451 - lava_callback - INFO - Starting async job submit for node 69c1f3d10efaf21d93106b79 2026-03-26 02:16:02,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:16:02,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f3d10efaf21d93106b79 HTTP/1.1" 200 None 2026-03-26 02:16:02,468 - lava_callback - DEBUG - Processing node 69c1f3d10efaf21d93106b79: baseline-arm64-clabbe 2026-03-26 02:16:02,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f3d10efaf21d93106b79 2026-03-26 02:16:02,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:16:02,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f3d10efaf21d93106b79 HTTP/1.1" 200 None 2026-03-26 02:16:02,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:16:02,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:16:02,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:16:02,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f3d10efaf21d93106b79 HTTP/1.1" 200 None 2026-03-26 02:16:02,544 - lava_callback - ERROR - Error processing callback for node 69c1f3d10efaf21d93106b79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:17:14,705 - lava_callback - INFO - Received callback for node 69c1f41d0efaf21d93106ba2 INFO: 172.21.0.1:51532 - "POST /node/69c1f41d0efaf21d93106ba2 HTTP/1.0" 202 Accepted 2026-03-26 02:17:14,711 - lava_callback - INFO - Starting async job submit for node 69c1f41d0efaf21d93106ba2 2026-03-26 02:17:14,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:17:14,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f41d0efaf21d93106ba2 HTTP/1.1" 200 None 2026-03-26 02:17:14,737 - lava_callback - DEBUG - Processing node 69c1f41d0efaf21d93106ba2: baseline-arm64-baylibre 2026-03-26 02:17:14,737 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f41d0efaf21d93106ba2 2026-03-26 02:17:14,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:17:14,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f41d0efaf21d93106ba2 HTTP/1.1" 200 None 2026-03-26 02:17:14,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:17:14,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:17:14,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:17:14,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f41d0efaf21d93106ba2 HTTP/1.1" 200 None 2026-03-26 02:17:14,820 - lava_callback - ERROR - Error processing callback for node 69c1f41d0efaf21d93106ba2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41864 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:18:42,582 - lava_callback - INFO - Received callback for node 69c1f47c0efaf21d93106bc9 INFO: 172.21.0.1:38248 - "POST /node/69c1f47c0efaf21d93106bc9 HTTP/1.0" 202 Accepted 2026-03-26 02:18:42,587 - lava_callback - INFO - Starting async job submit for node 69c1f47c0efaf21d93106bc9 2026-03-26 02:18:42,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:18:42,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f47c0efaf21d93106bc9 HTTP/1.1" 200 None 2026-03-26 02:18:42,609 - lava_callback - DEBUG - Processing node 69c1f47c0efaf21d93106bc9: baseline-arm64-clabbe 2026-03-26 02:18:42,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f47c0efaf21d93106bc9 2026-03-26 02:18:42,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:18:42,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f47c0efaf21d93106bc9 HTTP/1.1" 200 None 2026-03-26 02:18:42,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:18:42,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:18:42,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:18:42,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f47c0efaf21d93106bc9 HTTP/1.1" 200 None 2026-03-26 02:18:42,683 - lava_callback - ERROR - Error processing callback for node 69c1f47c0efaf21d93106bc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52088 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:19:03,236 - lava_callback - INFO - Received callback for node 69c4935d590270ad7b6d9955 INFO: 172.21.0.1:43980 - "POST /node/69c4935d590270ad7b6d9955 HTTP/1.0" 202 Accepted 2026-03-26 02:19:03,242 - lava_callback - INFO - Starting async job submit for node 69c4935d590270ad7b6d9955 2026-03-26 02:19:03,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:19:03,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4935d590270ad7b6d9955 HTTP/1.1" 200 None 2026-03-26 02:19:03,315 - lava_callback - DEBUG - Processing node 69c4935d590270ad7b6d9955: ltp-smoke 2026-03-26 02:19:03,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:19:03,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:19:03,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4935d590270ad7b6d9955/log.txt.gz 2026-03-26 02:19:04,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:19:04,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:19:04,615 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4935d590270ad7b6d9955/lava-logs.yaml 2026-03-26 02:19:04,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:19:04,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:19:04,804 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c4935d590270ad7b6d9955/lava_callback.json.gz 2026-03-26 02:19:04,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:19:04,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4935d590270ad7b6d9955 HTTP/1.1" 200 None 2026-03-26 02:19:04,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:19:04,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b41590270ad7b6d6994 HTTP/1.1" 200 None 2026-03-26 02:19:04,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:19:04,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4935d590270ad7b6d9955 HTTP/1.1" 200 None 2026-03-26 02:19:05,092 - lava_callback - INFO - Completed processing callback for node 69c4935d590270ad7b6d9955 INFO: 172.21.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:19:30,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:19:30,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:20:14,635 - lava_callback - INFO - Received callback for node 69c1f4cb0efaf21d93106be6 INFO: 172.21.0.1:51496 - "POST /node/69c1f4cb0efaf21d93106be6 HTTP/1.0" 202 Accepted 2026-03-26 02:20:14,641 - lava_callback - INFO - Starting async job submit for node 69c1f4cb0efaf21d93106be6 2026-03-26 02:20:14,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:14,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f4cb0efaf21d93106be6 HTTP/1.1" 200 None 2026-03-26 02:20:14,666 - lava_callback - DEBUG - Processing node 69c1f4cb0efaf21d93106be6: baseline-arm64-baylibre 2026-03-26 02:20:14,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f4cb0efaf21d93106be6 2026-03-26 02:20:14,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:14,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f4cb0efaf21d93106be6 HTTP/1.1" 200 None 2026-03-26 02:20:14,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:14,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:20:14,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:14,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f4cb0efaf21d93106be6 HTTP/1.1" 200 None 2026-03-26 02:20:14,734 - lava_callback - ERROR - Error processing callback for node 69c1f4cb0efaf21d93106be6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51508 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:20:25,220 - lava_callback - INFO - Received callback for node 69c493b1590270ad7b6d996d INFO: 172.21.0.1:51864 - "POST /node/69c493b1590270ad7b6d996d HTTP/1.0" 202 Accepted 2026-03-26 02:20:25,227 - lava_callback - INFO - Starting async job submit for node 69c493b1590270ad7b6d996d 2026-03-26 02:20:25,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:25,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c493b1590270ad7b6d996d HTTP/1.1" 200 None 2026-03-26 02:20:25,295 - lava_callback - DEBUG - Processing node 69c493b1590270ad7b6d996d: baseline-arm 2026-03-26 02:20:25,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:20:26,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:20:26,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c493b1590270ad7b6d996d/log.txt.gz 2026-03-26 02:20:26,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:51880 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:20:26,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:20:26,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c493b1590270ad7b6d996d/lava-logs.yaml 2026-03-26 02:20:26,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:20:26,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:20:26,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c493b1590270ad7b6d996d/lava_callback.json.gz DEBUG: Setup failed for 69c493b1590270ad7b6d996d. Transitting job node result: incomplete -> fail 2026-03-26 02:20:27,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:27,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c493b1590270ad7b6d996d HTTP/1.1" 200 None 2026-03-26 02:20:27,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:27,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 02:20:27,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:27,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c493b1590270ad7b6d996d HTTP/1.1" 200 None 2026-03-26 02:20:27,173 - lava_callback - INFO - Completed processing callback for node 69c493b1590270ad7b6d996d 2026-03-26 02:20:30,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:20:30,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50380 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:22:22,640 - lava_callback - INFO - Received callback for node 69c1f54f0efaf21d93106c1e INFO: 172.21.0.1:56526 - "POST /node/69c1f54f0efaf21d93106c1e HTTP/1.0" 202 Accepted 2026-03-26 02:22:22,646 - lava_callback - INFO - Starting async job submit for node 69c1f54f0efaf21d93106c1e 2026-03-26 02:22:22,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:22:22,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f54f0efaf21d93106c1e HTTP/1.1" 200 None 2026-03-26 02:22:22,698 - lava_callback - DEBUG - Processing node 69c1f54f0efaf21d93106c1e: baseline-arm64-clabbe 2026-03-26 02:22:22,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f54f0efaf21d93106c1e 2026-03-26 02:22:22,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:22:22,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f54f0efaf21d93106c1e HTTP/1.1" 200 None 2026-03-26 02:22:22,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:22:22,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:22:22,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:22:22,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f54f0efaf21d93106c1e HTTP/1.1" 200 None 2026-03-26 02:22:22,778 - lava_callback - ERROR - Error processing callback for node 69c1f54f0efaf21d93106c1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36642 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:24:06,699 - lava_callback - INFO - Received callback for node 69c49433590270ad7b6d9987 INFO: 172.21.0.1:36650 - "POST /node/69c49433590270ad7b6d9987 HTTP/1.0" 202 Accepted 2026-03-26 02:24:06,745 - lava_callback - INFO - Starting async job submit for node 69c49433590270ad7b6d9987 2026-03-26 02:24:06,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:06,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49433590270ad7b6d9987 HTTP/1.1" 200 None 2026-03-26 02:24:06,803 - lava_callback - DEBUG - Processing node 69c49433590270ad7b6d9987: baseline-arm-pengutronix 2026-03-26 02:24:06,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:24:07,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:24:07,058 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c49433590270ad7b6d9987/log.txt.gz 2026-03-26 02:24:07,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:24:07,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:24:07,315 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c49433590270ad7b6d9987/lava-logs.yaml 2026-03-26 02:24:07,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:24:07,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:24:07,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c49433590270ad7b6d9987/lava_callback.json.gz DEBUG: Setup failed for 69c49433590270ad7b6d9987. Transitting job node result: incomplete -> fail 2026-03-26 02:24:07,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:07,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49433590270ad7b6d9987 HTTP/1.1" 200 None 2026-03-26 02:24:07,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:07,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-26 02:24:07,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:07,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49433590270ad7b6d9987 HTTP/1.1" 200 None 2026-03-26 02:24:07,767 - lava_callback - INFO - Completed processing callback for node 69c49433590270ad7b6d9987 INFO: 172.21.0.1:49470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40878 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:24:30,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:30,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38062 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:24:55,163 - lava_callback - INFO - Received callback for node 69c1f5f20efaf21d93106c6c INFO: 172.21.0.1:38072 - "POST /node/69c1f5f20efaf21d93106c6c HTTP/1.0" 202 Accepted 2026-03-26 02:24:55,168 - lava_callback - INFO - Starting async job submit for node 69c1f5f20efaf21d93106c6c 2026-03-26 02:24:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:55,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f5f20efaf21d93106c6c HTTP/1.1" 200 None 2026-03-26 02:24:55,221 - lava_callback - DEBUG - Processing node 69c1f5f20efaf21d93106c6c: baseline-arm64-baylibre 2026-03-26 02:24:55,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f5f20efaf21d93106c6c 2026-03-26 02:24:55,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:55,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f5f20efaf21d93106c6c HTTP/1.1" 200 None 2026-03-26 02:24:55,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:55,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:24:55,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:24:55,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f5f20efaf21d93106c6c HTTP/1.1" 200 None 2026-03-26 02:24:55,278 - lava_callback - ERROR - Error processing callback for node 69c1f5f20efaf21d93106c6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:26:35,167 - lava_callback - INFO - Received callback for node 69c1f64b0efaf21d93106c91 INFO: 172.21.0.1:33478 - "POST /node/69c1f64b0efaf21d93106c91 HTTP/1.0" 202 Accepted 2026-03-26 02:26:35,172 - lava_callback - INFO - Starting async job submit for node 69c1f64b0efaf21d93106c91 2026-03-26 02:26:35,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:26:35,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f64b0efaf21d93106c91 HTTP/1.1" 200 None 2026-03-26 02:26:35,188 - lava_callback - DEBUG - Processing node 69c1f64b0efaf21d93106c91: baseline-arm64-baylibre 2026-03-26 02:26:35,188 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f64b0efaf21d93106c91 2026-03-26 02:26:35,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:26:35,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f64b0efaf21d93106c91 HTTP/1.1" 200 None 2026-03-26 02:26:35,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:26:35,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:26:35,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:26:35,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f64b0efaf21d93106c91 HTTP/1.1" 200 None 2026-03-26 02:26:35,246 - lava_callback - ERROR - Error processing callback for node 69c1f64b0efaf21d93106c91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:29:55,185 - lava_callback - INFO - Received callback for node 69c1f71d0efaf21d93106ce8 INFO: 172.21.0.1:35168 - "POST /node/69c1f71d0efaf21d93106ce8 HTTP/1.0" 202 Accepted 2026-03-26 02:29:55,190 - lava_callback - INFO - Starting async job submit for node 69c1f71d0efaf21d93106ce8 2026-03-26 02:29:55,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:29:55,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f71d0efaf21d93106ce8 HTTP/1.1" 200 None 2026-03-26 02:29:55,220 - lava_callback - DEBUG - Processing node 69c1f71d0efaf21d93106ce8: baseline-arm64-baylibre 2026-03-26 02:29:55,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f71d0efaf21d93106ce8 2026-03-26 02:29:55,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:29:55,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f71d0efaf21d93106ce8 HTTP/1.1" 200 None 2026-03-26 02:29:55,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:29:55,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:29:55,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:29:55,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f71d0efaf21d93106ce8 HTTP/1.1" 200 None 2026-03-26 02:29:55,307 - lava_callback - ERROR - Error processing callback for node 69c1f71d0efaf21d93106ce8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33464 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:31:35,319 - lava_callback - INFO - Received callback for node 69c1f7770efaf21d93106d0d INFO: 172.21.0.1:42492 - "POST /node/69c1f7770efaf21d93106d0d HTTP/1.0" 202 Accepted 2026-03-26 02:31:35,324 - lava_callback - INFO - Starting async job submit for node 69c1f7770efaf21d93106d0d 2026-03-26 02:31:35,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:31:35,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f7770efaf21d93106d0d HTTP/1.1" 200 None 2026-03-26 02:31:35,514 - lava_callback - DEBUG - Processing node 69c1f7770efaf21d93106d0d: baseline-arm64-baylibre 2026-03-26 02:31:35,514 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f7770efaf21d93106d0d 2026-03-26 02:31:35,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:31:35,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f7770efaf21d93106d0d HTTP/1.1" 200 None 2026-03-26 02:31:35,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:31:35,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:31:35,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:31:35,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f7770efaf21d93106d0d HTTP/1.1" 200 None 2026-03-26 02:31:35,626 - lava_callback - ERROR - Error processing callback for node 69c1f7770efaf21d93106d0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33516 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:33:03,164 - lava_callback - INFO - Received callback for node 69c1f7d40efaf21d93106d33 INFO: 172.21.0.1:39506 - "POST /node/69c1f7d40efaf21d93106d33 HTTP/1.0" 202 Accepted 2026-03-26 02:33:03,169 - lava_callback - INFO - Starting async job submit for node 69c1f7d40efaf21d93106d33 2026-03-26 02:33:03,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:03,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f7d40efaf21d93106d33 HTTP/1.1" 200 None 2026-03-26 02:33:03,195 - lava_callback - DEBUG - Processing node 69c1f7d40efaf21d93106d33: baseline-arm64-clabbe 2026-03-26 02:33:03,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f7d40efaf21d93106d33 2026-03-26 02:33:03,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:03,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f7d40efaf21d93106d33 HTTP/1.1" 200 None 2026-03-26 02:33:03,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:03,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:33:03,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:03,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f7d40efaf21d93106d33 HTTP/1.1" 200 None 2026-03-26 02:33:03,302 - lava_callback - ERROR - Error processing callback for node 69c1f7d40efaf21d93106d33: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58754 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:33:27,332 - lava_callback - INFO - Received callback for node 69c49645590270ad7b6d99b2 INFO: 172.21.0.1:58762 - "POST /node/69c49645590270ad7b6d99b2 HTTP/1.0" 202 Accepted 2026-03-26 02:33:27,340 - lava_callback - INFO - Starting async job submit for node 69c49645590270ad7b6d99b2 2026-03-26 02:33:27,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:27,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49645590270ad7b6d99b2 HTTP/1.1" 200 None 2026-03-26 02:33:27,420 - lava_callback - DEBUG - Processing node 69c49645590270ad7b6d99b2: baseline-nfs-arm64 2026-03-26 02:33:28,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:33:28,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:33:28,746 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49645590270ad7b6d99b2/log.txt.gz 2026-03-26 02:33:29,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:33:29,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:33:29,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49645590270ad7b6d99b2/lava-logs.yaml 2026-03-26 02:33:29,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:33:29,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:33:29,809 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49645590270ad7b6d99b2/lava_callback.json.gz DEBUG: Setup failed for 69c49645590270ad7b6d99b2. Transitting job node result: incomplete -> fail 2026-03-26 02:33:29,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:29,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49645590270ad7b6d99b2 HTTP/1.1" 200 None 2026-03-26 02:33:29,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:29,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:33:29,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:29,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49645590270ad7b6d99b2 HTTP/1.1" 200 None 2026-03-26 02:33:30,097 - lava_callback - INFO - Completed processing callback for node 69c49645590270ad7b6d99b2 2026-03-26 02:33:30,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:33:30,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58672 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:34:43,174 - lava_callback - INFO - Received callback for node 69c1f82d0efaf21d93106d5d INFO: 172.21.0.1:59464 - "POST /node/69c1f82d0efaf21d93106d5d HTTP/1.0" 202 Accepted 2026-03-26 02:34:43,181 - lava_callback - INFO - Starting async job submit for node 69c1f82d0efaf21d93106d5d 2026-03-26 02:34:43,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:34:43,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f82d0efaf21d93106d5d HTTP/1.1" 200 None 2026-03-26 02:34:43,201 - lava_callback - DEBUG - Processing node 69c1f82d0efaf21d93106d5d: baseline-arm64-clabbe 2026-03-26 02:34:43,202 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f82d0efaf21d93106d5d 2026-03-26 02:34:43,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:34:43,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f82d0efaf21d93106d5d HTTP/1.1" 200 None 2026-03-26 02:34:43,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:34:43,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:34:43,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:34:43,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f82d0efaf21d93106d5d HTTP/1.1" 200 None 2026-03-26 02:34:43,256 - lava_callback - ERROR - Error processing callback for node 69c1f82d0efaf21d93106d5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59080 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:36:03,201 - lava_callback - INFO - Received callback for node 69c1f8840efaf21d93106d82 INFO: 172.21.0.1:36686 - "POST /node/69c1f8840efaf21d93106d82 HTTP/1.0" 202 Accepted 2026-03-26 02:36:03,207 - lava_callback - INFO - Starting async job submit for node 69c1f8840efaf21d93106d82 2026-03-26 02:36:03,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:36:03,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f8840efaf21d93106d82 HTTP/1.1" 200 None 2026-03-26 02:36:03,246 - lava_callback - DEBUG - Processing node 69c1f8840efaf21d93106d82: baseline-arm64-clabbe 2026-03-26 02:36:03,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f8840efaf21d93106d82 2026-03-26 02:36:03,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:36:03,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f8840efaf21d93106d82 HTTP/1.1" 200 None 2026-03-26 02:36:03,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:36:03,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:36:03,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:36:03,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f8840efaf21d93106d82 HTTP/1.1" 200 None 2026-03-26 02:36:03,307 - lava_callback - ERROR - Error processing callback for node 69c1f8840efaf21d93106d82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41542 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:37:23,301 - lava_callback - INFO - Received callback for node 69c1f8db0efaf21d93106d9f INFO: 172.21.0.1:37918 - "POST /node/69c1f8db0efaf21d93106d9f HTTP/1.0" 202 Accepted 2026-03-26 02:37:23,307 - lava_callback - INFO - Starting async job submit for node 69c1f8db0efaf21d93106d9f 2026-03-26 02:37:23,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:37:23,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f8db0efaf21d93106d9f HTTP/1.1" 200 None 2026-03-26 02:37:23,339 - lava_callback - DEBUG - Processing node 69c1f8db0efaf21d93106d9f: baseline-arm64-clabbe 2026-03-26 02:37:23,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f8db0efaf21d93106d9f 2026-03-26 02:37:23,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:37:23,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f8db0efaf21d93106d9f HTTP/1.1" 200 None 2026-03-26 02:37:23,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:37:23,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:37:23,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:37:23,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f8db0efaf21d93106d9f HTTP/1.1" 200 None 2026-03-26 02:37:23,394 - lava_callback - ERROR - Error processing callback for node 69c1f8db0efaf21d93106d9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44430 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:38:43,339 - lava_callback - INFO - Received callback for node 69c1f9300efaf21d93106dcb INFO: 172.21.0.1:35864 - "POST /node/69c1f9300efaf21d93106dcb HTTP/1.0" 202 Accepted 2026-03-26 02:38:43,345 - lava_callback - INFO - Starting async job submit for node 69c1f9300efaf21d93106dcb 2026-03-26 02:38:43,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:38:43,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f9300efaf21d93106dcb HTTP/1.1" 200 None 2026-03-26 02:38:43,366 - lava_callback - DEBUG - Processing node 69c1f9300efaf21d93106dcb: baseline-arm64-clabbe 2026-03-26 02:38:43,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f9300efaf21d93106dcb 2026-03-26 02:38:43,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:38:43,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f9300efaf21d93106dcb HTTP/1.1" 200 None 2026-03-26 02:38:43,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:38:43,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:38:43,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:38:43,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f9300efaf21d93106dcb HTTP/1.1" 200 None 2026-03-26 02:38:43,430 - lava_callback - ERROR - Error processing callback for node 69c1f9300efaf21d93106dcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50294 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:40:23,433 - lava_callback - INFO - Received callback for node 69c1f98a0efaf21d93106dfd INFO: 172.21.0.1:48910 - "POST /node/69c1f98a0efaf21d93106dfd HTTP/1.0" 202 Accepted 2026-03-26 02:40:23,438 - lava_callback - INFO - Starting async job submit for node 69c1f98a0efaf21d93106dfd 2026-03-26 02:40:23,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:40:23,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f98a0efaf21d93106dfd HTTP/1.1" 200 None 2026-03-26 02:40:23,461 - lava_callback - DEBUG - Processing node 69c1f98a0efaf21d93106dfd: baseline-arm64-clabbe 2026-03-26 02:40:23,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f98a0efaf21d93106dfd 2026-03-26 02:40:23,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:40:23,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f98a0efaf21d93106dfd HTTP/1.1" 200 None 2026-03-26 02:40:23,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:40:23,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 02:40:23,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:40:23,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f98a0efaf21d93106dfd HTTP/1.1" 200 None 2026-03-26 02:40:23,519 - lava_callback - ERROR - Error processing callback for node 69c1f98a0efaf21d93106dfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36276 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:41:43,477 - lava_callback - INFO - Received callback for node 69c1f9e00efaf21d93106e26 INFO: 172.21.0.1:50564 - "POST /node/69c1f9e00efaf21d93106e26 HTTP/1.0" 202 Accepted 2026-03-26 02:41:43,484 - lava_callback - INFO - Starting async job submit for node 69c1f9e00efaf21d93106e26 2026-03-26 02:41:43,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:41:43,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f9e00efaf21d93106e26 HTTP/1.1" 200 None 2026-03-26 02:41:43,499 - lava_callback - DEBUG - Processing node 69c1f9e00efaf21d93106e26: baseline-arm64-clabbe 2026-03-26 02:41:43,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1f9e00efaf21d93106e26 2026-03-26 02:41:43,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:41:43,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1f9e00efaf21d93106e26 HTTP/1.1" 200 None 2026-03-26 02:41:43,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:41:43,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:41:43,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:41:43,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1f9e00efaf21d93106e26 HTTP/1.1" 200 None 2026-03-26 02:41:43,573 - lava_callback - ERROR - Error processing callback for node 69c1f9e00efaf21d93106e26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:44:43,655 - lava_callback - INFO - Received callback for node 69c1fa860efaf21d93106e75 INFO: 172.21.0.1:54462 - "POST /node/69c1fa860efaf21d93106e75 HTTP/1.0" 202 Accepted 2026-03-26 02:44:43,665 - lava_callback - INFO - Starting async job submit for node 69c1fa860efaf21d93106e75 2026-03-26 02:44:43,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:44:43,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fa860efaf21d93106e75 HTTP/1.1" 200 None 2026-03-26 02:44:43,708 - lava_callback - DEBUG - Processing node 69c1fa860efaf21d93106e75: baseline-arm64-clabbe 2026-03-26 02:44:43,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fa860efaf21d93106e75 2026-03-26 02:44:43,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:44:43,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fa860efaf21d93106e75 HTTP/1.1" 200 None 2026-03-26 02:44:43,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:44:43,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:44:43,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:44:43,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fa860efaf21d93106e75 HTTP/1.1" 200 None 2026-03-26 02:44:43,826 - lava_callback - ERROR - Error processing callback for node 69c1fa860efaf21d93106e75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49202 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:46:03,678 - lava_callback - INFO - Received callback for node 69c1fad90efaf21d93106e90 INFO: 172.21.0.1:44964 - "POST /node/69c1fad90efaf21d93106e90 HTTP/1.0" 202 Accepted 2026-03-26 02:46:03,683 - lava_callback - INFO - Starting async job submit for node 69c1fad90efaf21d93106e90 2026-03-26 02:46:03,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:46:03,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fad90efaf21d93106e90 HTTP/1.1" 200 None 2026-03-26 02:46:03,700 - lava_callback - DEBUG - Processing node 69c1fad90efaf21d93106e90: baseline-arm64-clabbe 2026-03-26 02:46:03,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fad90efaf21d93106e90 2026-03-26 02:46:03,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:46:03,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fad90efaf21d93106e90 HTTP/1.1" 200 None 2026-03-26 02:46:03,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:46:03,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:46:03,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:46:03,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fad90efaf21d93106e90 HTTP/1.1" 200 None 2026-03-26 02:46:03,758 - lava_callback - ERROR - Error processing callback for node 69c1fad90efaf21d93106e90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32928 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:47:23,777 - lava_callback - INFO - Received callback for node 69c1fb2c0efaf21d93106eb3 INFO: 172.21.0.1:60410 - "POST /node/69c1fb2c0efaf21d93106eb3 HTTP/1.0" 202 Accepted 2026-03-26 02:47:23,783 - lava_callback - INFO - Starting async job submit for node 69c1fb2c0efaf21d93106eb3 2026-03-26 02:47:23,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:47:23,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fb2c0efaf21d93106eb3 HTTP/1.1" 200 None 2026-03-26 02:47:23,799 - lava_callback - DEBUG - Processing node 69c1fb2c0efaf21d93106eb3: baseline-arm64-clabbe 2026-03-26 02:47:23,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fb2c0efaf21d93106eb3 2026-03-26 02:47:23,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:47:23,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fb2c0efaf21d93106eb3 HTTP/1.1" 200 None 2026-03-26 02:47:23,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:47:23,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:47:23,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:47:23,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fb2c0efaf21d93106eb3 HTTP/1.1" 200 None 2026-03-26 02:47:23,853 - lava_callback - ERROR - Error processing callback for node 69c1fb2c0efaf21d93106eb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47858 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:48:43,808 - lava_callback - INFO - Received callback for node 69c1fb7f0efaf21d93106edb INFO: 172.21.0.1:59714 - "POST /node/69c1fb7f0efaf21d93106edb HTTP/1.0" 202 Accepted 2026-03-26 02:48:43,814 - lava_callback - INFO - Starting async job submit for node 69c1fb7f0efaf21d93106edb 2026-03-26 02:48:43,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:48:43,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fb7f0efaf21d93106edb HTTP/1.1" 200 None 2026-03-26 02:48:43,839 - lava_callback - DEBUG - Processing node 69c1fb7f0efaf21d93106edb: baseline-arm64-clabbe 2026-03-26 02:48:43,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fb7f0efaf21d93106edb 2026-03-26 02:48:43,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:48:43,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fb7f0efaf21d93106edb HTTP/1.1" 200 None 2026-03-26 02:48:43,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:48:43,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:48:43,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:48:43,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fb7f0efaf21d93106edb HTTP/1.1" 200 None 2026-03-26 02:48:43,902 - lava_callback - ERROR - Error processing callback for node 69c1fb7f0efaf21d93106edb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59648 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:51:24,070 - lava_callback - INFO - Received callback for node 69c1fc260efaf21d93106f29 INFO: 172.21.0.1:53548 - "POST /node/69c1fc260efaf21d93106f29 HTTP/1.0" 202 Accepted 2026-03-26 02:51:24,076 - lava_callback - INFO - Starting async job submit for node 69c1fc260efaf21d93106f29 2026-03-26 02:51:24,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:51:24,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fc260efaf21d93106f29 HTTP/1.1" 200 None 2026-03-26 02:51:24,092 - lava_callback - DEBUG - Processing node 69c1fc260efaf21d93106f29: baseline-arm64-clabbe 2026-03-26 02:51:24,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fc260efaf21d93106f29 2026-03-26 02:51:24,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:51:24,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fc260efaf21d93106f29 HTTP/1.1" 200 None 2026-03-26 02:51:24,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:51:24,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:51:24,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:51:24,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fc260efaf21d93106f29 HTTP/1.1" 200 None 2026-03-26 02:51:24,169 - lava_callback - ERROR - Error processing callback for node 69c1fc260efaf21d93106f29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50074 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:53:04,073 - lava_callback - INFO - Received callback for node 69c1fc7a0efaf21d93106f57 INFO: 172.21.0.1:54926 - "POST /node/69c1fc7a0efaf21d93106f57 HTTP/1.0" 202 Accepted 2026-03-26 02:53:04,079 - lava_callback - INFO - Starting async job submit for node 69c1fc7a0efaf21d93106f57 2026-03-26 02:53:04,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:53:04,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fc7a0efaf21d93106f57 HTTP/1.1" 200 None 2026-03-26 02:53:04,095 - lava_callback - DEBUG - Processing node 69c1fc7a0efaf21d93106f57: baseline-arm64-clabbe 2026-03-26 02:53:04,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fc7a0efaf21d93106f57 2026-03-26 02:53:04,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:53:04,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fc7a0efaf21d93106f57 HTTP/1.1" 200 None 2026-03-26 02:53:04,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:53:04,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-26 02:53:04,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:53:04,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fc7a0efaf21d93106f57 HTTP/1.1" 200 None 2026-03-26 02:53:04,158 - lava_callback - ERROR - Error processing callback for node 69c1fc7a0efaf21d93106f57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60828 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:54:29,018 - lava_callback - INFO - Received callback for node 69c49b38590270ad7b6d9a1d INFO: 172.21.0.1:60842 - "POST /node/69c49b38590270ad7b6d9a1d HTTP/1.0" 202 Accepted 2026-03-26 02:54:29,025 - lava_callback - INFO - Starting async job submit for node 69c49b38590270ad7b6d9a1d 2026-03-26 02:54:29,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:54:29,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49b38590270ad7b6d9a1d HTTP/1.1" 200 None 2026-03-26 02:54:29,110 - lava_callback - DEBUG - Processing node 69c49b38590270ad7b6d9a1d: baseline-nfs-arm64 2026-03-26 02:54:30,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:54:30,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:54:30,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49b38590270ad7b6d9a1d/log.txt.gz 2026-03-26 02:54:31,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:54:31,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:54:31,461 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49b38590270ad7b6d9a1d/lava-logs.yaml 2026-03-26 02:54:31,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 02:54:31,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 02:54:31,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c49b38590270ad7b6d9a1d/lava_callback.json.gz DEBUG: Setup failed for 69c49b38590270ad7b6d9a1d. Transitting job node result: incomplete -> fail 2026-03-26 02:54:31,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:54:31,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49b38590270ad7b6d9a1d HTTP/1.1" 200 None 2026-03-26 02:54:31,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:54:31,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 02:54:31,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:54:32,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49b38590270ad7b6d9a1d HTTP/1.1" 200 None 2026-03-26 02:54:32,122 - lava_callback - INFO - Completed processing callback for node 69c49b38590270ad7b6d9a1d INFO: 172.21.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33886 - "GET / HTTP/1.0" 200 OK 2026-03-26 02:55:00,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 02:55:00,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47040 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:00:37,000 - lava_callback - INFO - Received callback for node 69c1fe420efaf21d93107022 INFO: 172.21.0.1:47056 - "POST /node/69c1fe420efaf21d93107022 HTTP/1.0" 202 Accepted 2026-03-26 03:00:37,006 - lava_callback - INFO - Starting async job submit for node 69c1fe420efaf21d93107022 2026-03-26 03:00:37,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:00:37,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fe420efaf21d93107022 HTTP/1.1" 200 None 2026-03-26 03:00:37,022 - lava_callback - DEBUG - Processing node 69c1fe420efaf21d93107022: baseline-arm-baylibre 2026-03-26 03:00:37,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fe420efaf21d93107022 2026-03-26 03:00:37,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:00:37,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fe420efaf21d93107022 HTTP/1.1" 200 None 2026-03-26 03:00:37,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:00:37,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-26 03:00:37,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:00:37,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fe420efaf21d93107022 HTTP/1.1" 200 None 2026-03-26 03:00:37,083 - lava_callback - ERROR - Error processing callback for node 69c1fe420efaf21d93107022: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43862 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:01:16,915 - lava_callback - INFO - Received callback for node 69c1fe6a0efaf21d93107033 INFO: 172.21.0.1:43868 - "POST /node/69c1fe6a0efaf21d93107033 HTTP/1.0" 202 Accepted 2026-03-26 03:01:16,921 - lava_callback - INFO - Starting async job submit for node 69c1fe6a0efaf21d93107033 2026-03-26 03:01:16,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:01:16,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fe6a0efaf21d93107033 HTTP/1.1" 200 None 2026-03-26 03:01:16,963 - lava_callback - DEBUG - Processing node 69c1fe6a0efaf21d93107033: baseline-arm-baylibre 2026-03-26 03:01:16,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1fe6a0efaf21d93107033 2026-03-26 03:01:16,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:01:16,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1fe6a0efaf21d93107033 HTTP/1.1" 200 None 2026-03-26 03:01:16,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:01:16,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-26 03:01:16,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:01:17,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1fe6a0efaf21d93107033 HTTP/1.1" 200 None 2026-03-26 03:01:17,037 - lava_callback - ERROR - Error processing callback for node 69c1fe6a0efaf21d93107033: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49914 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:06:44,822 - lava_callback - INFO - Received callback for node 69c1ffb10efaf21d931070c9 INFO: 172.21.0.1:59734 - "POST /node/69c1ffb10efaf21d931070c9 HTTP/1.0" 202 Accepted 2026-03-26 03:06:44,827 - lava_callback - INFO - Starting async job submit for node 69c1ffb10efaf21d931070c9 2026-03-26 03:06:44,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:06:44,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ffb10efaf21d931070c9 HTTP/1.1" 200 None 2026-03-26 03:06:44,844 - lava_callback - DEBUG - Processing node 69c1ffb10efaf21d931070c9: baseline-arm-clabbe 2026-03-26 03:06:44,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1ffb10efaf21d931070c9 2026-03-26 03:06:44,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:06:44,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ffb10efaf21d931070c9 HTTP/1.1" 200 None 2026-03-26 03:06:44,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:06:44,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-26 03:06:44,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:06:44,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ffb10efaf21d931070c9 HTTP/1.1" 200 None 2026-03-26 03:06:44,915 - lava_callback - ERROR - Error processing callback for node 69c1ffb10efaf21d931070c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51814 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:07:24,577 - lava_callback - INFO - Received callback for node 69c1ffda0efaf21d931070d8 INFO: 172.21.0.1:53612 - "POST /node/69c1ffda0efaf21d931070d8 HTTP/1.0" 202 Accepted 2026-03-26 03:07:24,582 - lava_callback - INFO - Starting async job submit for node 69c1ffda0efaf21d931070d8 2026-03-26 03:07:24,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:07:24,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ffda0efaf21d931070d8 HTTP/1.1" 200 None 2026-03-26 03:07:24,610 - lava_callback - DEBUG - Processing node 69c1ffda0efaf21d931070d8: baseline-arm-clabbe 2026-03-26 03:07:24,610 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c1ffda0efaf21d931070d8 2026-03-26 03:07:24,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:07:24,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1ffda0efaf21d931070d8 HTTP/1.1" 200 None 2026-03-26 03:07:24,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:07:24,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-26 03:07:24,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:07:24,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1ffda0efaf21d931070d8 HTTP/1.1" 200 None 2026-03-26 03:07:24,699 - lava_callback - ERROR - Error processing callback for node 69c1ffda0efaf21d931070d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32986 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:08:04,689 - lava_callback - INFO - Received callback for node 69c200050efaf21d931070ea INFO: 172.21.0.1:47028 - "POST /node/69c200050efaf21d931070ea HTTP/1.0" 202 Accepted 2026-03-26 03:08:04,694 - lava_callback - INFO - Starting async job submit for node 69c200050efaf21d931070ea 2026-03-26 03:08:04,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:04,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c200050efaf21d931070ea HTTP/1.1" 200 None 2026-03-26 03:08:04,711 - lava_callback - DEBUG - Processing node 69c200050efaf21d931070ea: baseline-arm-clabbe 2026-03-26 03:08:04,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c200050efaf21d931070ea 2026-03-26 03:08:04,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:04,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c200050efaf21d931070ea HTTP/1.1" 200 None 2026-03-26 03:08:04,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:04,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-26 03:08:04,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:04,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c200050efaf21d931070ea HTTP/1.1" 200 None 2026-03-26 03:08:04,774 - lava_callback - ERROR - Error processing callback for node 69c200050efaf21d931070ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36346 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:08:44,646 - lava_callback - INFO - Received callback for node 69c200310efaf21d931070fd INFO: 172.21.0.1:38462 - "POST /node/69c200310efaf21d931070fd HTTP/1.0" 202 Accepted 2026-03-26 03:08:44,651 - lava_callback - INFO - Starting async job submit for node 69c200310efaf21d931070fd 2026-03-26 03:08:44,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:44,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c200310efaf21d931070fd HTTP/1.1" 200 None 2026-03-26 03:08:44,667 - lava_callback - DEBUG - Processing node 69c200310efaf21d931070fd: baseline-arm-clabbe 2026-03-26 03:08:44,668 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c200310efaf21d931070fd 2026-03-26 03:08:44,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:44,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c200310efaf21d931070fd HTTP/1.1" 200 None 2026-03-26 03:08:44,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:44,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-26 03:08:44,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:08:44,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c200310efaf21d931070fd HTTP/1.1" 200 None 2026-03-26 03:08:44,728 - lava_callback - ERROR - Error processing callback for node 69c200310efaf21d931070fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:09:24,697 - lava_callback - INFO - Received callback for node 69c2005c0efaf21d9310710e INFO: 172.21.0.1:38564 - "POST /node/69c2005c0efaf21d9310710e HTTP/1.0" 202 Accepted 2026-03-26 03:09:24,702 - lava_callback - INFO - Starting async job submit for node 69c2005c0efaf21d9310710e 2026-03-26 03:09:24,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:09:24,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2005c0efaf21d9310710e HTTP/1.1" 200 None 2026-03-26 03:09:24,718 - lava_callback - DEBUG - Processing node 69c2005c0efaf21d9310710e: baseline-arm-clabbe 2026-03-26 03:09:24,718 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2005c0efaf21d9310710e 2026-03-26 03:09:24,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:09:24,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2005c0efaf21d9310710e HTTP/1.1" 200 None 2026-03-26 03:09:24,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:09:24,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-26 03:09:24,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:09:24,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2005c0efaf21d9310710e HTTP/1.1" 200 None 2026-03-26 03:09:24,774 - lava_callback - ERROR - Error processing callback for node 69c2005c0efaf21d9310710e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37608 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:16:17,932 - lava_callback - INFO - Received callback for node 69c201ed0efaf21d93107199 INFO: 172.21.0.1:37614 - "POST /node/69c201ed0efaf21d93107199 HTTP/1.0" 202 Accepted 2026-03-26 03:16:17,938 - lava_callback - INFO - Starting async job submit for node 69c201ed0efaf21d93107199 2026-03-26 03:16:17,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:16:17,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c201ed0efaf21d93107199 HTTP/1.1" 200 None 2026-03-26 03:16:17,957 - lava_callback - DEBUG - Processing node 69c201ed0efaf21d93107199: baseline-arm64-baylibre 2026-03-26 03:16:17,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c201ed0efaf21d93107199 2026-03-26 03:16:17,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:16:17,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c201ed0efaf21d93107199 HTTP/1.1" 200 None 2026-03-26 03:16:17,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:16:17,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:16:18,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:16:18,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c201ed0efaf21d93107199 HTTP/1.1" 200 None 2026-03-26 03:16:18,033 - lava_callback - ERROR - Error processing callback for node 69c201ed0efaf21d93107199: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32912 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:17:37,998 - lava_callback - INFO - Received callback for node 69c2023f0efaf21d931071b3 INFO: 172.21.0.1:32926 - "POST /node/69c2023f0efaf21d931071b3 HTTP/1.0" 202 Accepted 2026-03-26 03:17:38,003 - lava_callback - INFO - Starting async job submit for node 69c2023f0efaf21d931071b3 2026-03-26 03:17:38,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:17:38,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2023f0efaf21d931071b3 HTTP/1.1" 200 None 2026-03-26 03:17:38,019 - lava_callback - DEBUG - Processing node 69c2023f0efaf21d931071b3: baseline-arm64-baylibre 2026-03-26 03:17:38,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2023f0efaf21d931071b3 2026-03-26 03:17:38,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:17:38,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2023f0efaf21d931071b3 HTTP/1.1" 200 None 2026-03-26 03:17:38,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:17:38,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:17:38,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:17:38,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2023f0efaf21d931071b3 HTTP/1.1" 200 None 2026-03-26 03:17:38,073 - lava_callback - ERROR - Error processing callback for node 69c2023f0efaf21d931071b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54574 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:20:18,073 - lava_callback - INFO - Received callback for node 69c202e50efaf21d9310720a INFO: 172.21.0.1:54588 - "POST /node/69c202e50efaf21d9310720a HTTP/1.0" 202 Accepted 2026-03-26 03:20:18,079 - lava_callback - INFO - Starting async job submit for node 69c202e50efaf21d9310720a 2026-03-26 03:20:18,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:20:18,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c202e50efaf21d9310720a HTTP/1.1" 200 None 2026-03-26 03:20:18,133 - lava_callback - DEBUG - Processing node 69c202e50efaf21d9310720a: baseline-arm64-baylibre 2026-03-26 03:20:18,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c202e50efaf21d9310720a 2026-03-26 03:20:18,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:20:18,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c202e50efaf21d9310720a HTTP/1.1" 200 None 2026-03-26 03:20:18,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:20:18,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:20:18,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:20:18,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c202e50efaf21d9310720a HTTP/1.1" 200 None 2026-03-26 03:20:18,276 - lava_callback - ERROR - Error processing callback for node 69c202e50efaf21d9310720a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35860 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:21:29,528 - lava_callback - INFO - Received callback for node 69c48870590270ad7b6d957f INFO: 172.21.0.1:35866 - "POST /node/69c48870590270ad7b6d957f HTTP/1.0" 202 Accepted 2026-03-26 03:21:29,534 - lava_callback - INFO - Starting async job submit for node 69c48870590270ad7b6d957f 2026-03-26 03:21:29,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:21:29,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48870590270ad7b6d957f HTTP/1.1" 200 None 2026-03-26 03:21:29,571 - lava_callback - DEBUG - Processing node 69c48870590270ad7b6d957f: baseline-arm64-broonie 2026-03-26 03:21:29,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 03:21:29,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 03:21:29,952 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48870590270ad7b6d957f/log.txt.gz 2026-03-26 03:21:30,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 03:21:30,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 03:21:30,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48870590270ad7b6d957f/lava-logs.yaml 2026-03-26 03:21:30,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 03:21:30,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 03:21:30,435 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48870590270ad7b6d957f/lava_callback.json.gz 2026-03-26 03:21:30,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:21:30,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48870590270ad7b6d957f HTTP/1.1" 200 None 2026-03-26 03:21:30,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:21:30,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 03:21:30,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:21:30,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48870590270ad7b6d957f HTTP/1.1" 200 None 2026-03-26 03:21:30,538 - lava_callback - INFO - Completed processing callback for node 69c48870590270ad7b6d957f 2026-03-26 03:21:30,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:21:30,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57452 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:22:58,429 - lava_callback - INFO - Received callback for node 69c203890efaf21d93107255 INFO: 172.21.0.1:57458 - "POST /node/69c203890efaf21d93107255 HTTP/1.0" 202 Accepted 2026-03-26 03:22:58,435 - lava_callback - INFO - Starting async job submit for node 69c203890efaf21d93107255 2026-03-26 03:22:58,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:22:58,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c203890efaf21d93107255 HTTP/1.1" 200 None 2026-03-26 03:22:58,463 - lava_callback - DEBUG - Processing node 69c203890efaf21d93107255: baseline-arm64-baylibre 2026-03-26 03:22:58,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c203890efaf21d93107255 2026-03-26 03:22:58,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:22:58,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c203890efaf21d93107255 HTTP/1.1" 200 None 2026-03-26 03:22:58,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:22:58,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:22:58,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:22:58,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c203890efaf21d93107255 HTTP/1.1" 200 None 2026-03-26 03:22:58,517 - lava_callback - ERROR - Error processing callback for node 69c203890efaf21d93107255: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36462 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:24:18,219 - lava_callback - INFO - Received callback for node 69c203dd0efaf21d93107285 INFO: 172.21.0.1:36478 - "POST /node/69c203dd0efaf21d93107285 HTTP/1.0" 202 Accepted 2026-03-26 03:24:18,224 - lava_callback - INFO - Starting async job submit for node 69c203dd0efaf21d93107285 2026-03-26 03:24:18,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:24:18,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c203dd0efaf21d93107285 HTTP/1.1" 200 None 2026-03-26 03:24:18,241 - lava_callback - DEBUG - Processing node 69c203dd0efaf21d93107285: baseline-arm64-baylibre 2026-03-26 03:24:18,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c203dd0efaf21d93107285 2026-03-26 03:24:18,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:24:18,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c203dd0efaf21d93107285 HTTP/1.1" 200 None 2026-03-26 03:24:18,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:24:18,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:24:18,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:24:18,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c203dd0efaf21d93107285 HTTP/1.1" 200 None 2026-03-26 03:24:18,303 - lava_callback - ERROR - Error processing callback for node 69c203dd0efaf21d93107285: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45378 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:25:38,335 - lava_callback - INFO - Received callback for node 69c2042e0efaf21d931072b3 INFO: 172.21.0.1:45390 - "POST /node/69c2042e0efaf21d931072b3 HTTP/1.0" 202 Accepted 2026-03-26 03:25:38,340 - lava_callback - INFO - Starting async job submit for node 69c2042e0efaf21d931072b3 2026-03-26 03:25:38,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:25:38,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2042e0efaf21d931072b3 HTTP/1.1" 200 None 2026-03-26 03:25:38,356 - lava_callback - DEBUG - Processing node 69c2042e0efaf21d931072b3: baseline-arm64-baylibre 2026-03-26 03:25:38,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2042e0efaf21d931072b3 2026-03-26 03:25:38,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:25:38,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2042e0efaf21d931072b3 HTTP/1.1" 200 None 2026-03-26 03:25:38,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:25:38,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:25:38,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:25:38,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2042e0efaf21d931072b3 HTTP/1.1" 200 None 2026-03-26 03:25:38,421 - lava_callback - ERROR - Error processing callback for node 69c2042e0efaf21d931072b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:27:18,442 - lava_callback - INFO - Received callback for node 69c204840efaf21d931072e0 INFO: 172.21.0.1:34838 - "POST /node/69c204840efaf21d931072e0 HTTP/1.0" 202 Accepted 2026-03-26 03:27:18,447 - lava_callback - INFO - Starting async job submit for node 69c204840efaf21d931072e0 2026-03-26 03:27:18,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:27:18,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c204840efaf21d931072e0 HTTP/1.1" 200 None 2026-03-26 03:27:18,469 - lava_callback - DEBUG - Processing node 69c204840efaf21d931072e0: baseline-arm64-baylibre 2026-03-26 03:27:18,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c204840efaf21d931072e0 2026-03-26 03:27:18,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:27:18,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c204840efaf21d931072e0 HTTP/1.1" 200 None 2026-03-26 03:27:18,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:27:18,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:27:18,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:27:18,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c204840efaf21d931072e0 HTTP/1.1" 200 None 2026-03-26 03:27:18,541 - lava_callback - ERROR - Error processing callback for node 69c204840efaf21d931072e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46330 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:28:38,670 - lava_callback - INFO - Received callback for node 69c204d60efaf21d9310730b INFO: 172.21.0.1:46338 - "POST /node/69c204d60efaf21d9310730b HTTP/1.0" 202 Accepted 2026-03-26 03:28:38,676 - lava_callback - INFO - Starting async job submit for node 69c204d60efaf21d9310730b 2026-03-26 03:28:38,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:28:38,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c204d60efaf21d9310730b HTTP/1.1" 200 None 2026-03-26 03:28:38,692 - lava_callback - DEBUG - Processing node 69c204d60efaf21d9310730b: baseline-arm64-baylibre 2026-03-26 03:28:38,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c204d60efaf21d9310730b 2026-03-26 03:28:38,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:28:38,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c204d60efaf21d9310730b HTTP/1.1" 200 None 2026-03-26 03:28:38,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:28:38,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:28:38,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:28:38,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c204d60efaf21d9310730b HTTP/1.1" 200 None 2026-03-26 03:28:38,746 - lava_callback - ERROR - Error processing callback for node 69c204d60efaf21d9310730b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38438 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:29:58,574 - lava_callback - INFO - Received callback for node 69c205290efaf21d9310732b INFO: 172.21.0.1:38446 - "POST /node/69c205290efaf21d9310732b HTTP/1.0" 202 Accepted 2026-03-26 03:29:58,580 - lava_callback - INFO - Starting async job submit for node 69c205290efaf21d9310732b 2026-03-26 03:29:58,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:29:58,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c205290efaf21d9310732b HTTP/1.1" 200 None 2026-03-26 03:29:58,596 - lava_callback - DEBUG - Processing node 69c205290efaf21d9310732b: baseline-arm64-baylibre 2026-03-26 03:29:58,597 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c205290efaf21d9310732b 2026-03-26 03:29:58,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:29:58,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c205290efaf21d9310732b HTTP/1.1" 200 None 2026-03-26 03:29:58,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:29:58,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:29:58,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:29:58,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c205290efaf21d9310732b HTTP/1.1" 200 None 2026-03-26 03:29:58,674 - lava_callback - ERROR - Error processing callback for node 69c205290efaf21d9310732b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43040 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:31:25,678 - lava_callback - INFO - Received callback for node 69c205840efaf21d93107351 INFO: 172.21.0.1:37804 - "POST /node/69c205840efaf21d93107351 HTTP/1.0" 202 Accepted 2026-03-26 03:31:25,683 - lava_callback - INFO - Starting async job submit for node 69c205840efaf21d93107351 2026-03-26 03:31:25,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:31:25,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c205840efaf21d93107351 HTTP/1.1" 200 None 2026-03-26 03:31:25,699 - lava_callback - DEBUG - Processing node 69c205840efaf21d93107351: baseline-arm64-clabbe 2026-03-26 03:31:25,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c205840efaf21d93107351 2026-03-26 03:31:25,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:31:25,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c205840efaf21d93107351 HTTP/1.1" 200 None 2026-03-26 03:31:25,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:31:25,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:31:25,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:31:25,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c205840efaf21d93107351 HTTP/1.1" 200 None 2026-03-26 03:31:25,758 - lava_callback - ERROR - Error processing callback for node 69c205840efaf21d93107351: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51680 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:32:45,704 - lava_callback - INFO - Received callback for node 69c205d60efaf21d93107376 INFO: 172.21.0.1:51048 - "POST /node/69c205d60efaf21d93107376 HTTP/1.0" 202 Accepted 2026-03-26 03:32:45,709 - lava_callback - INFO - Starting async job submit for node 69c205d60efaf21d93107376 2026-03-26 03:32:45,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:32:45,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c205d60efaf21d93107376 HTTP/1.1" 200 None 2026-03-26 03:32:45,725 - lava_callback - DEBUG - Processing node 69c205d60efaf21d93107376: baseline-arm64-clabbe 2026-03-26 03:32:45,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c205d60efaf21d93107376 2026-03-26 03:32:45,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:32:45,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c205d60efaf21d93107376 HTTP/1.1" 200 None 2026-03-26 03:32:45,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:32:45,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:32:45,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:32:45,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c205d60efaf21d93107376 HTTP/1.1" 200 None 2026-03-26 03:32:45,791 - lava_callback - ERROR - Error processing callback for node 69c205d60efaf21d93107376: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60910 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:33:58,849 - lava_callback - INFO - Received callback for node 69c206220efaf21d931073a3 INFO: 172.21.0.1:60916 - "POST /node/69c206220efaf21d931073a3 HTTP/1.0" 202 Accepted 2026-03-26 03:33:58,855 - lava_callback - INFO - Starting async job submit for node 69c206220efaf21d931073a3 2026-03-26 03:33:58,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:33:58,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c206220efaf21d931073a3 HTTP/1.1" 200 None 2026-03-26 03:33:58,871 - lava_callback - DEBUG - Processing node 69c206220efaf21d931073a3: baseline-arm64-baylibre 2026-03-26 03:33:58,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c206220efaf21d931073a3 2026-03-26 03:33:58,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:33:58,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c206220efaf21d931073a3 HTTP/1.1" 200 None 2026-03-26 03:33:58,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:33:58,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:33:58,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:33:58,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c206220efaf21d931073a3 HTTP/1.1" 200 None 2026-03-26 03:33:58,969 - lava_callback - ERROR - Error processing callback for node 69c206220efaf21d931073a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:35:45,873 - lava_callback - INFO - Received callback for node 69c206800efaf21d931073cd INFO: 172.21.0.1:40986 - "POST /node/69c206800efaf21d931073cd HTTP/1.0" 202 Accepted 2026-03-26 03:35:45,878 - lava_callback - INFO - Starting async job submit for node 69c206800efaf21d931073cd 2026-03-26 03:35:45,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:35:45,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c206800efaf21d931073cd HTTP/1.1" 200 None 2026-03-26 03:35:45,909 - lava_callback - DEBUG - Processing node 69c206800efaf21d931073cd: baseline-arm64-clabbe 2026-03-26 03:35:45,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c206800efaf21d931073cd 2026-03-26 03:35:45,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:35:45,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c206800efaf21d931073cd HTTP/1.1" 200 None 2026-03-26 03:35:45,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:35:45,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:35:45,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:35:46,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c206800efaf21d931073cd HTTP/1.1" 200 None 2026-03-26 03:35:46,005 - lava_callback - ERROR - Error processing callback for node 69c206800efaf21d931073cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:36:58,908 - lava_callback - INFO - Received callback for node 69c206cd0efaf21d931073eb INFO: 172.21.0.1:43160 - "POST /node/69c206cd0efaf21d931073eb HTTP/1.0" 202 Accepted 2026-03-26 03:36:58,913 - lava_callback - INFO - Starting async job submit for node 69c206cd0efaf21d931073eb 2026-03-26 03:36:58,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:36:58,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c206cd0efaf21d931073eb HTTP/1.1" 200 None 2026-03-26 03:36:58,930 - lava_callback - DEBUG - Processing node 69c206cd0efaf21d931073eb: baseline-arm64-baylibre 2026-03-26 03:36:58,930 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c206cd0efaf21d931073eb 2026-03-26 03:36:58,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:36:58,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c206cd0efaf21d931073eb HTTP/1.1" 200 None 2026-03-26 03:36:58,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:36:58,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:36:58,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:36:58,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c206cd0efaf21d931073eb HTTP/1.1" 200 None 2026-03-26 03:36:58,990 - lava_callback - ERROR - Error processing callback for node 69c206cd0efaf21d931073eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:38:25,971 - lava_callback - INFO - Received callback for node 69c2072c0efaf21d93107415 INFO: 172.21.0.1:52124 - "POST /node/69c2072c0efaf21d93107415 HTTP/1.0" 202 Accepted 2026-03-26 03:38:25,977 - lava_callback - INFO - Starting async job submit for node 69c2072c0efaf21d93107415 2026-03-26 03:38:25,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:38:26,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2072c0efaf21d93107415 HTTP/1.1" 200 None 2026-03-26 03:38:26,018 - lava_callback - DEBUG - Processing node 69c2072c0efaf21d93107415: baseline-arm64-clabbe 2026-03-26 03:38:26,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2072c0efaf21d93107415 2026-03-26 03:38:26,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:38:26,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2072c0efaf21d93107415 HTTP/1.1" 200 None 2026-03-26 03:38:26,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:38:26,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:38:26,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:38:26,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2072c0efaf21d93107415 HTTP/1.1" 200 None 2026-03-26 03:38:26,080 - lava_callback - ERROR - Error processing callback for node 69c2072c0efaf21d93107415: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47278 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:39:59,177 - lava_callback - INFO - Received callback for node 69c207790efaf21d9310743d INFO: 172.21.0.1:47280 - "POST /node/69c207790efaf21d9310743d HTTP/1.0" 202 Accepted 2026-03-26 03:39:59,183 - lava_callback - INFO - Starting async job submit for node 69c207790efaf21d9310743d 2026-03-26 03:39:59,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:39:59,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c207790efaf21d9310743d HTTP/1.1" 200 None 2026-03-26 03:39:59,200 - lava_callback - DEBUG - Processing node 69c207790efaf21d9310743d: baseline-arm64-baylibre 2026-03-26 03:39:59,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c207790efaf21d9310743d 2026-03-26 03:39:59,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:39:59,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c207790efaf21d9310743d HTTP/1.1" 200 None 2026-03-26 03:39:59,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:39:59,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:39:59,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:39:59,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c207790efaf21d9310743d HTTP/1.1" 200 None 2026-03-26 03:39:59,305 - lava_callback - ERROR - Error processing callback for node 69c207790efaf21d9310743d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32982 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:41:26,081 - lava_callback - INFO - Received callback for node 69c207d70efaf21d93107464 INFO: 172.21.0.1:58730 - "POST /node/69c207d70efaf21d93107464 HTTP/1.0" 202 Accepted 2026-03-26 03:41:26,087 - lava_callback - INFO - Starting async job submit for node 69c207d70efaf21d93107464 2026-03-26 03:41:26,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:41:26,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c207d70efaf21d93107464 HTTP/1.1" 200 None 2026-03-26 03:41:26,125 - lava_callback - DEBUG - Processing node 69c207d70efaf21d93107464: baseline-arm64-clabbe 2026-03-26 03:41:26,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c207d70efaf21d93107464 2026-03-26 03:41:26,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:41:26,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c207d70efaf21d93107464 HTTP/1.1" 200 None 2026-03-26 03:41:26,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:41:26,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:41:26,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:41:26,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c207d70efaf21d93107464 HTTP/1.1" 200 None 2026-03-26 03:41:26,235 - lava_callback - ERROR - Error processing callback for node 69c207d70efaf21d93107464: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42914 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:42:46,139 - lava_callback - INFO - Received callback for node 69c2082a0efaf21d9310748a INFO: 172.21.0.1:41760 - "POST /node/69c2082a0efaf21d9310748a HTTP/1.0" 202 Accepted 2026-03-26 03:42:46,146 - lava_callback - INFO - Starting async job submit for node 69c2082a0efaf21d9310748a 2026-03-26 03:42:46,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:42:46,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2082a0efaf21d9310748a HTTP/1.1" 200 None 2026-03-26 03:42:46,189 - lava_callback - DEBUG - Processing node 69c2082a0efaf21d9310748a: baseline-arm64-clabbe 2026-03-26 03:42:46,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2082a0efaf21d9310748a 2026-03-26 03:42:46,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:42:46,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2082a0efaf21d9310748a HTTP/1.1" 200 None 2026-03-26 03:42:46,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:42:46,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:42:46,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:42:46,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2082a0efaf21d9310748a HTTP/1.1" 200 None 2026-03-26 03:42:46,288 - lava_callback - ERROR - Error processing callback for node 69c2082a0efaf21d9310748a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:44:06,139 - lava_callback - INFO - Received callback for node 69c2087d0efaf21d931074a8 INFO: 172.21.0.1:38872 - "POST /node/69c2087d0efaf21d931074a8 HTTP/1.0" 202 Accepted 2026-03-26 03:44:06,145 - lava_callback - INFO - Starting async job submit for node 69c2087d0efaf21d931074a8 2026-03-26 03:44:06,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:44:06,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2087d0efaf21d931074a8 HTTP/1.1" 200 None 2026-03-26 03:44:06,161 - lava_callback - DEBUG - Processing node 69c2087d0efaf21d931074a8: baseline-arm64-clabbe 2026-03-26 03:44:06,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2087d0efaf21d931074a8 2026-03-26 03:44:06,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:44:06,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2087d0efaf21d931074a8 HTTP/1.1" 200 None 2026-03-26 03:44:06,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:44:06,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-26 03:44:06,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:44:06,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2087d0efaf21d931074a8 HTTP/1.1" 200 None 2026-03-26 03:44:06,230 - lava_callback - ERROR - Error processing callback for node 69c2087d0efaf21d931074a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46158 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:45:26,243 - lava_callback - INFO - Received callback for node 69c208d10efaf21d931074c8 INFO: 172.21.0.1:56966 - "POST /node/69c208d10efaf21d931074c8 HTTP/1.0" 202 Accepted 2026-03-26 03:45:26,248 - lava_callback - INFO - Starting async job submit for node 69c208d10efaf21d931074c8 2026-03-26 03:45:26,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:45:26,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c208d10efaf21d931074c8 HTTP/1.1" 200 None 2026-03-26 03:45:26,273 - lava_callback - DEBUG - Processing node 69c208d10efaf21d931074c8: baseline-arm64-clabbe 2026-03-26 03:45:26,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c208d10efaf21d931074c8 2026-03-26 03:45:26,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:45:26,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c208d10efaf21d931074c8 HTTP/1.1" 200 None 2026-03-26 03:45:26,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:45:26,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:45:26,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:45:26,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c208d10efaf21d931074c8 HTTP/1.1" 200 None 2026-03-26 03:45:26,338 - lava_callback - ERROR - Error processing callback for node 69c208d10efaf21d931074c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38772 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:47:06,280 - lava_callback - INFO - Received callback for node 69c209280efaf21d931074df INFO: 172.21.0.1:37476 - "POST /node/69c209280efaf21d931074df HTTP/1.0" 202 Accepted 2026-03-26 03:47:06,286 - lava_callback - INFO - Starting async job submit for node 69c209280efaf21d931074df 2026-03-26 03:47:06,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:47:06,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c209280efaf21d931074df HTTP/1.1" 200 None 2026-03-26 03:47:06,302 - lava_callback - DEBUG - Processing node 69c209280efaf21d931074df: baseline-arm64-clabbe 2026-03-26 03:47:06,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c209280efaf21d931074df 2026-03-26 03:47:06,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:47:06,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c209280efaf21d931074df HTTP/1.1" 200 None 2026-03-26 03:47:06,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:47:06,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:47:06,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:47:06,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c209280efaf21d931074df HTTP/1.1" 200 None 2026-03-26 03:47:06,365 - lava_callback - ERROR - Error processing callback for node 69c209280efaf21d931074df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42476 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:48:26,333 - lava_callback - INFO - Received callback for node 69c209800efaf21d931074fe INFO: 172.21.0.1:56566 - "POST /node/69c209800efaf21d931074fe HTTP/1.0" 202 Accepted 2026-03-26 03:48:26,338 - lava_callback - INFO - Starting async job submit for node 69c209800efaf21d931074fe 2026-03-26 03:48:26,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:48:26,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c209800efaf21d931074fe HTTP/1.1" 200 None 2026-03-26 03:48:26,354 - lava_callback - DEBUG - Processing node 69c209800efaf21d931074fe: baseline-arm64-clabbe 2026-03-26 03:48:26,354 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c209800efaf21d931074fe 2026-03-26 03:48:26,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:48:26,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c209800efaf21d931074fe HTTP/1.1" 200 None 2026-03-26 03:48:26,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:48:26,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:48:26,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:48:26,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c209800efaf21d931074fe HTTP/1.1" 200 None 2026-03-26 03:48:26,414 - lava_callback - ERROR - Error processing callback for node 69c209800efaf21d931074fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:49:46,424 - lava_callback - INFO - Received callback for node 69c209d30efaf21d93107515 INFO: 172.21.0.1:56310 - "POST /node/69c209d30efaf21d93107515 HTTP/1.0" 202 Accepted 2026-03-26 03:49:46,430 - lava_callback - INFO - Starting async job submit for node 69c209d30efaf21d93107515 2026-03-26 03:49:46,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:49:46,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c209d30efaf21d93107515 HTTP/1.1" 200 None 2026-03-26 03:49:46,446 - lava_callback - DEBUG - Processing node 69c209d30efaf21d93107515: baseline-arm64-clabbe 2026-03-26 03:49:46,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c209d30efaf21d93107515 2026-03-26 03:49:46,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:49:46,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c209d30efaf21d93107515 HTTP/1.1" 200 None 2026-03-26 03:49:46,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:49:46,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:49:46,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:49:46,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c209d30efaf21d93107515 HTTP/1.1" 200 None 2026-03-26 03:49:46,500 - lava_callback - ERROR - Error processing callback for node 69c209d30efaf21d93107515: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:51:06,469 - lava_callback - INFO - Received callback for node 69c20a270efaf21d9310753a INFO: 172.21.0.1:36308 - "POST /node/69c20a270efaf21d9310753a HTTP/1.0" 202 Accepted 2026-03-26 03:51:06,475 - lava_callback - INFO - Starting async job submit for node 69c20a270efaf21d9310753a 2026-03-26 03:51:06,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:51:06,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c20a270efaf21d9310753a HTTP/1.1" 200 None 2026-03-26 03:51:06,491 - lava_callback - DEBUG - Processing node 69c20a270efaf21d9310753a: baseline-arm64-clabbe 2026-03-26 03:51:06,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c20a270efaf21d9310753a 2026-03-26 03:51:06,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:51:06,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c20a270efaf21d9310753a HTTP/1.1" 200 None 2026-03-26 03:51:06,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 03:51:06,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-26 03:51:06,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:36310 - "GET / HTTP/1.0" 200 OK 2026-03-26 03:51:06,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c20a270efaf21d9310753a HTTP/1.1" 200 None 2026-03-26 03:51:06,549 - lava_callback - ERROR - Error processing callback for node 69c20a270efaf21d9310753a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45454 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:47:49,076 - lava_callback - INFO - Received callback for node 69c2176d0efaf21d93107622 INFO: 172.21.0.1:45466 - "POST /node/69c2176d0efaf21d93107622 HTTP/1.0" 202 Accepted 2026-03-26 04:47:49,081 - lava_callback - INFO - Starting async job submit for node 69c2176d0efaf21d93107622 2026-03-26 04:47:49,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:47:49,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2176d0efaf21d93107622 HTTP/1.1" 200 None 2026-03-26 04:47:49,098 - lava_callback - DEBUG - Processing node 69c2176d0efaf21d93107622: baseline-arm-clabbe 2026-03-26 04:47:49,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2176d0efaf21d93107622 2026-03-26 04:47:49,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:47:49,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2176d0efaf21d93107622 HTTP/1.1" 200 None 2026-03-26 04:47:49,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:47:49,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 04:47:49,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:47:49,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2176d0efaf21d93107622 HTTP/1.1" 200 None 2026-03-26 04:47:49,199 - lava_callback - ERROR - Error processing callback for node 69c2176d0efaf21d93107622: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35992 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:48:29,018 - lava_callback - INFO - Received callback for node 69c217940efaf21d93107633 INFO: 172.21.0.1:36002 - "POST /node/69c217940efaf21d93107633 HTTP/1.0" 202 Accepted 2026-03-26 04:48:29,024 - lava_callback - INFO - Starting async job submit for node 69c217940efaf21d93107633 2026-03-26 04:48:29,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:48:29,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c217940efaf21d93107633 HTTP/1.1" 200 None 2026-03-26 04:48:29,040 - lava_callback - DEBUG - Processing node 69c217940efaf21d93107633: baseline-arm-clabbe 2026-03-26 04:48:29,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c217940efaf21d93107633 2026-03-26 04:48:29,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:48:29,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c217940efaf21d93107633 HTTP/1.1" 200 None 2026-03-26 04:48:29,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:48:29,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 04:48:29,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:48:29,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c217940efaf21d93107633 HTTP/1.1" 200 None 2026-03-26 04:48:29,126 - lava_callback - ERROR - Error processing callback for node 69c217940efaf21d93107633: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42848 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:49:09,087 - lava_callback - INFO - Received callback for node 69c217bc0efaf21d93107645 INFO: 172.21.0.1:42864 - "POST /node/69c217bc0efaf21d93107645 HTTP/1.0" 202 Accepted 2026-03-26 04:49:09,093 - lava_callback - INFO - Starting async job submit for node 69c217bc0efaf21d93107645 2026-03-26 04:49:09,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:09,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c217bc0efaf21d93107645 HTTP/1.1" 200 None 2026-03-26 04:49:09,113 - lava_callback - DEBUG - Processing node 69c217bc0efaf21d93107645: baseline-arm-clabbe 2026-03-26 04:49:09,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c217bc0efaf21d93107645 2026-03-26 04:49:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:09,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c217bc0efaf21d93107645 HTTP/1.1" 200 None 2026-03-26 04:49:09,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:09,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 04:49:09,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:09,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c217bc0efaf21d93107645 HTTP/1.1" 200 None 2026-03-26 04:49:09,183 - lava_callback - ERROR - Error processing callback for node 69c217bc0efaf21d93107645: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59544 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:49:49,071 - lava_callback - INFO - Received callback for node 69c217e60efaf21d93107658 INFO: 172.21.0.1:59552 - "POST /node/69c217e60efaf21d93107658 HTTP/1.0" 202 Accepted 2026-03-26 04:49:49,077 - lava_callback - INFO - Starting async job submit for node 69c217e60efaf21d93107658 2026-03-26 04:49:49,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:49,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c217e60efaf21d93107658 HTTP/1.1" 200 None 2026-03-26 04:49:49,097 - lava_callback - DEBUG - Processing node 69c217e60efaf21d93107658: baseline-arm-clabbe 2026-03-26 04:49:49,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c217e60efaf21d93107658 2026-03-26 04:49:49,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:49,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c217e60efaf21d93107658 HTTP/1.1" 200 None 2026-03-26 04:49:49,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:49,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 04:49:49,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:49:49,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c217e60efaf21d93107658 HTTP/1.1" 200 None 2026-03-26 04:49:49,157 - lava_callback - ERROR - Error processing callback for node 69c217e60efaf21d93107658: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59766 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:50:29,123 - lava_callback - INFO - Received callback for node 69c2180e0efaf21d93107671 INFO: 172.21.0.1:59770 - "POST /node/69c2180e0efaf21d93107671 HTTP/1.0" 202 Accepted 2026-03-26 04:50:29,128 - lava_callback - INFO - Starting async job submit for node 69c2180e0efaf21d93107671 2026-03-26 04:50:29,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:50:29,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2180e0efaf21d93107671 HTTP/1.1" 200 None 2026-03-26 04:50:29,146 - lava_callback - DEBUG - Processing node 69c2180e0efaf21d93107671: baseline-arm-clabbe 2026-03-26 04:50:29,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2180e0efaf21d93107671 2026-03-26 04:50:29,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:50:29,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2180e0efaf21d93107671 HTTP/1.1" 200 None 2026-03-26 04:50:29,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:50:29,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-26 04:50:29,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:50:29,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2180e0efaf21d93107671 HTTP/1.1" 200 None 2026-03-26 04:50:29,201 - lava_callback - ERROR - Error processing callback for node 69c2180e0efaf21d93107671: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39614 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:53:48,077 - lava_callback - INFO - Received callback for node 69c411a3944faaf0e7f60e39 INFO: 172.21.0.1:39622 - "POST /node/69c411a3944faaf0e7f60e39 HTTP/1.0" 202 Accepted 2026-03-26 04:53:48,083 - lava_callback - INFO - Starting async job submit for node 69c411a3944faaf0e7f60e39 2026-03-26 04:53:48,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:53:48,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c411a3944faaf0e7f60e39 HTTP/1.1" 200 None 2026-03-26 04:53:48,120 - lava_callback - DEBUG - Processing node 69c411a3944faaf0e7f60e39: baseline-arm64-broonie 2026-03-26 04:53:48,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:53:48,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:53:48,481 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a3944faaf0e7f60e39/log.txt.gz 2026-03-26 04:53:48,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:53:48,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:53:48,834 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a3944faaf0e7f60e39/lava-logs.yaml 2026-03-26 04:53:48,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:53:49,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:53:49,010 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a3944faaf0e7f60e39/lava_callback.json.gz 2026-03-26 04:53:49,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:53:49,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c411a3944faaf0e7f60e39 HTTP/1.1" 200 None 2026-03-26 04:53:49,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:53:49,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 04:53:49,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:53:49,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c411a3944faaf0e7f60e39 HTTP/1.1" 200 None 2026-03-26 04:53:49,105 - lava_callback - INFO - Completed processing callback for node 69c411a3944faaf0e7f60e39 INFO: 172.21.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:54:00,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:54:00,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48184 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:54:08,167 - lava_callback - INFO - Received callback for node 69c46cda944faaf0e7f62046 INFO: 172.21.0.1:48188 - "POST /node/69c46cda944faaf0e7f62046 HTTP/1.0" 202 Accepted 2026-03-26 04:54:08,173 - lava_callback - INFO - Starting async job submit for node 69c46cda944faaf0e7f62046 2026-03-26 04:54:08,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:54:08,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46cda944faaf0e7f62046 HTTP/1.1" 200 None 2026-03-26 04:54:08,227 - lava_callback - DEBUG - Processing node 69c46cda944faaf0e7f62046: baseline-arm64-broonie 2026-03-26 04:54:08,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:54:08,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:54:08,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c46cda944faaf0e7f62046/log.txt.gz 2026-03-26 04:54:08,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:54:08,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:54:08,896 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c46cda944faaf0e7f62046/lava-logs.yaml 2026-03-26 04:54:08,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:54:09,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:54:09,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c46cda944faaf0e7f62046/lava_callback.json.gz 2026-03-26 04:54:09,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:54:09,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c46cda944faaf0e7f62046 HTTP/1.1" 200 None 2026-03-26 04:54:09,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:54:09,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 04:54:09,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:54:09,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c46cda944faaf0e7f62046 HTTP/1.1" 200 None 2026-03-26 04:54:09,167 - lava_callback - INFO - Completed processing callback for node 69c46cda944faaf0e7f62046 INFO: 172.21.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:54:30,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:54:30,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59616 - "GET / HTTP/1.0" 200 OK 2026-03-26 04:55:27,784 - lava_callback - INFO - Received callback for node 69c4886e590270ad7b6d9578 INFO: 172.21.0.1:59628 - "POST /node/69c4886e590270ad7b6d9578 HTTP/1.0" 202 Accepted 2026-03-26 04:55:27,790 - lava_callback - INFO - Starting async job submit for node 69c4886e590270ad7b6d9578 2026-03-26 04:55:27,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:55:27,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4886e590270ad7b6d9578 HTTP/1.1" 200 None 2026-03-26 04:55:27,827 - lava_callback - DEBUG - Processing node 69c4886e590270ad7b6d9578: baseline-arm64-broonie 2026-03-26 04:55:28,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:55:28,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:55:28,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4886e590270ad7b6d9578/log.txt.gz 2026-03-26 04:55:28,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:55:28,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:55:28,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4886e590270ad7b6d9578/lava-logs.yaml 2026-03-26 04:55:28,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 04:55:28,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 04:55:28,786 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c4886e590270ad7b6d9578/lava_callback.json.gz 2026-03-26 04:55:28,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:55:28,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4886e590270ad7b6d9578 HTTP/1.1" 200 None 2026-03-26 04:55:28,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:55:28,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 04:55:28,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:55:28,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4886e590270ad7b6d9578 HTTP/1.1" 200 None 2026-03-26 04:55:28,882 - lava_callback - INFO - Completed processing callback for node 69c4886e590270ad7b6d9578 2026-03-26 04:55:30,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 04:55:30,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54470 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:00:43,453 - lava_callback - INFO - Received callback for node 69c21a790efaf21d93107683 INFO: 172.21.0.1:57458 - "POST /node/69c21a790efaf21d93107683 HTTP/1.0" 202 Accepted 2026-03-26 05:00:43,458 - lava_callback - INFO - Starting async job submit for node 69c21a790efaf21d93107683 2026-03-26 05:00:43,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:00:43,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21a790efaf21d93107683 HTTP/1.1" 200 None 2026-03-26 05:00:43,476 - lava_callback - DEBUG - Processing node 69c21a790efaf21d93107683: baseline-arm64-baylibre 2026-03-26 05:00:43,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21a790efaf21d93107683 2026-03-26 05:00:43,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:00:43,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21a790efaf21d93107683 HTTP/1.1" 200 None 2026-03-26 05:00:43,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:00:43,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:00:43,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:00:43,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21a790efaf21d93107683 HTTP/1.1" 200 None 2026-03-26 05:00:43,544 - lava_callback - ERROR - Error processing callback for node 69c21a790efaf21d93107683: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49080 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:02:23,559 - lava_callback - INFO - Received callback for node 69c21acf0efaf21d93107691 INFO: 172.21.0.1:38986 - "POST /node/69c21acf0efaf21d93107691 HTTP/1.0" 202 Accepted 2026-03-26 05:02:23,565 - lava_callback - INFO - Starting async job submit for node 69c21acf0efaf21d93107691 2026-03-26 05:02:23,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:02:23,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21acf0efaf21d93107691 HTTP/1.1" 200 None 2026-03-26 05:02:23,581 - lava_callback - DEBUG - Processing node 69c21acf0efaf21d93107691: baseline-arm64-baylibre 2026-03-26 05:02:23,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21acf0efaf21d93107691 2026-03-26 05:02:23,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:02:23,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21acf0efaf21d93107691 HTTP/1.1" 200 None 2026-03-26 05:02:23,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:02:23,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:02:23,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:02:23,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21acf0efaf21d93107691 HTTP/1.1" 200 None 2026-03-26 05:02:23,671 - lava_callback - ERROR - Error processing callback for node 69c21acf0efaf21d93107691: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55076 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:03:43,637 - lava_callback - INFO - Received callback for node 69c21b230efaf21d9310769f INFO: 172.21.0.1:58354 - "POST /node/69c21b230efaf21d9310769f HTTP/1.0" 202 Accepted 2026-03-26 05:03:43,643 - lava_callback - INFO - Starting async job submit for node 69c21b230efaf21d9310769f 2026-03-26 05:03:43,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:03:43,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21b230efaf21d9310769f HTTP/1.1" 200 None 2026-03-26 05:03:43,660 - lava_callback - DEBUG - Processing node 69c21b230efaf21d9310769f: baseline-arm64-baylibre 2026-03-26 05:03:43,660 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21b230efaf21d9310769f 2026-03-26 05:03:43,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:03:43,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21b230efaf21d9310769f HTTP/1.1" 200 None 2026-03-26 05:03:43,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:03:43,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:03:43,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:03:43,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21b230efaf21d9310769f HTTP/1.1" 200 None 2026-03-26 05:03:43,722 - lava_callback - ERROR - Error processing callback for node 69c21b230efaf21d9310769f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44700 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:05:03,731 - lava_callback - INFO - Received callback for node 69c21b780efaf21d931076ad INFO: 172.21.0.1:58256 - "POST /node/69c21b780efaf21d931076ad HTTP/1.0" 202 Accepted 2026-03-26 05:05:03,736 - lava_callback - INFO - Starting async job submit for node 69c21b780efaf21d931076ad 2026-03-26 05:05:03,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:05:03,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21b780efaf21d931076ad HTTP/1.1" 200 None 2026-03-26 05:05:03,754 - lava_callback - DEBUG - Processing node 69c21b780efaf21d931076ad: baseline-arm64-baylibre 2026-03-26 05:05:03,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21b780efaf21d931076ad 2026-03-26 05:05:03,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:05:03,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21b780efaf21d931076ad HTTP/1.1" 200 None 2026-03-26 05:05:03,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:05:03,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:05:03,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:05:03,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21b780efaf21d931076ad HTTP/1.1" 200 None 2026-03-26 05:05:03,813 - lava_callback - ERROR - Error processing callback for node 69c21b780efaf21d931076ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36572 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:06:43,898 - lava_callback - INFO - Received callback for node 69c21bce0efaf21d931076bb INFO: 172.21.0.1:45432 - "POST /node/69c21bce0efaf21d931076bb HTTP/1.0" 202 Accepted 2026-03-26 05:06:43,903 - lava_callback - INFO - Starting async job submit for node 69c21bce0efaf21d931076bb 2026-03-26 05:06:43,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:06:43,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21bce0efaf21d931076bb HTTP/1.1" 200 None 2026-03-26 05:06:43,920 - lava_callback - DEBUG - Processing node 69c21bce0efaf21d931076bb: baseline-arm64-baylibre 2026-03-26 05:06:43,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21bce0efaf21d931076bb 2026-03-26 05:06:43,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:06:43,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21bce0efaf21d931076bb HTTP/1.1" 200 None 2026-03-26 05:06:43,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:06:43,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:06:43,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:06:43,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21bce0efaf21d931076bb HTTP/1.1" 200 None 2026-03-26 05:06:43,983 - lava_callback - ERROR - Error processing callback for node 69c21bce0efaf21d931076bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60544 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:08:03,819 - lava_callback - INFO - Received callback for node 69c21c250efaf21d931076c4 INFO: 172.21.0.1:59336 - "POST /node/69c21c250efaf21d931076c4 HTTP/1.0" 202 Accepted 2026-03-26 05:08:03,825 - lava_callback - INFO - Starting async job submit for node 69c21c250efaf21d931076c4 2026-03-26 05:08:03,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:08:03,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21c250efaf21d931076c4 HTTP/1.1" 200 None 2026-03-26 05:08:03,840 - lava_callback - DEBUG - Processing node 69c21c250efaf21d931076c4: baseline-arm64-baylibre 2026-03-26 05:08:03,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21c250efaf21d931076c4 2026-03-26 05:08:03,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:08:03,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21c250efaf21d931076c4 HTTP/1.1" 200 None 2026-03-26 05:08:03,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:08:03,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:08:03,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:08:03,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21c250efaf21d931076c4 HTTP/1.1" 200 None 2026-03-26 05:08:03,897 - lava_callback - ERROR - Error processing callback for node 69c21c250efaf21d931076c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41968 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:09:23,900 - lava_callback - INFO - Received callback for node 69c21c7e0efaf21d931076d6 INFO: 172.21.0.1:33926 - "POST /node/69c21c7e0efaf21d931076d6 HTTP/1.0" 202 Accepted 2026-03-26 05:09:23,909 - lava_callback - INFO - Starting async job submit for node 69c21c7e0efaf21d931076d6 2026-03-26 05:09:23,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:09:23,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21c7e0efaf21d931076d6 HTTP/1.1" 200 None 2026-03-26 05:09:23,925 - lava_callback - DEBUG - Processing node 69c21c7e0efaf21d931076d6: baseline-arm64-baylibre 2026-03-26 05:09:23,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21c7e0efaf21d931076d6 2026-03-26 05:09:23,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:09:23,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21c7e0efaf21d931076d6 HTTP/1.1" 200 None 2026-03-26 05:09:23,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:09:23,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:09:23,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:09:23,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21c7e0efaf21d931076d6 HTTP/1.1" 200 None 2026-03-26 05:09:23,975 - lava_callback - ERROR - Error processing callback for node 69c21c7e0efaf21d931076d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55132 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:11:04,247 - lava_callback - INFO - Received callback for node 69c21cd20efaf21d931076ec INFO: 172.21.0.1:42766 - "POST /node/69c21cd20efaf21d931076ec HTTP/1.0" 202 Accepted 2026-03-26 05:11:04,252 - lava_callback - INFO - Starting async job submit for node 69c21cd20efaf21d931076ec 2026-03-26 05:11:04,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:11:04,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21cd20efaf21d931076ec HTTP/1.1" 200 None 2026-03-26 05:11:04,291 - lava_callback - DEBUG - Processing node 69c21cd20efaf21d931076ec: baseline-arm64-baylibre 2026-03-26 05:11:04,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21cd20efaf21d931076ec 2026-03-26 05:11:04,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:11:04,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21cd20efaf21d931076ec HTTP/1.1" 200 None 2026-03-26 05:11:04,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:11:04,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:11:04,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:11:04,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21cd20efaf21d931076ec HTTP/1.1" 200 None 2026-03-26 05:11:04,353 - lava_callback - ERROR - Error processing callback for node 69c21cd20efaf21d931076ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:12:24,085 - lava_callback - INFO - Received callback for node 69c21d2a0efaf21d9310770c INFO: 172.21.0.1:38302 - "POST /node/69c21d2a0efaf21d9310770c HTTP/1.0" 202 Accepted 2026-03-26 05:12:24,090 - lava_callback - INFO - Starting async job submit for node 69c21d2a0efaf21d9310770c 2026-03-26 05:12:24,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:12:24,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21d2a0efaf21d9310770c HTTP/1.1" 200 None 2026-03-26 05:12:24,112 - lava_callback - DEBUG - Processing node 69c21d2a0efaf21d9310770c: baseline-arm64-baylibre 2026-03-26 05:12:24,112 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21d2a0efaf21d9310770c 2026-03-26 05:12:24,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:12:24,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21d2a0efaf21d9310770c HTTP/1.1" 200 None 2026-03-26 05:12:24,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:12:24,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:12:24,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:12:24,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21d2a0efaf21d9310770c HTTP/1.1" 200 None 2026-03-26 05:12:24,169 - lava_callback - ERROR - Error processing callback for node 69c21d2a0efaf21d9310770c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45162 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:13:44,159 - lava_callback - INFO - Received callback for node 69c21d7e0efaf21d93107731 INFO: 172.21.0.1:35170 - "POST /node/69c21d7e0efaf21d93107731 HTTP/1.0" 202 Accepted 2026-03-26 05:13:44,165 - lava_callback - INFO - Starting async job submit for node 69c21d7e0efaf21d93107731 2026-03-26 05:13:44,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:13:44,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21d7e0efaf21d93107731 HTTP/1.1" 200 None 2026-03-26 05:13:44,181 - lava_callback - DEBUG - Processing node 69c21d7e0efaf21d93107731: baseline-arm64-baylibre 2026-03-26 05:13:44,181 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21d7e0efaf21d93107731 2026-03-26 05:13:44,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:13:44,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21d7e0efaf21d93107731 HTTP/1.1" 200 None 2026-03-26 05:13:44,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:13:44,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:13:44,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:13:44,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21d7e0efaf21d93107731 HTTP/1.1" 200 None 2026-03-26 05:13:44,241 - lava_callback - ERROR - Error processing callback for node 69c21d7e0efaf21d93107731: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42168 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:15:24,237 - lava_callback - INFO - Received callback for node 69c21ddb0efaf21d93107741 INFO: 172.21.0.1:49140 - "POST /node/69c21ddb0efaf21d93107741 HTTP/1.0" 202 Accepted 2026-03-26 05:15:24,243 - lava_callback - INFO - Starting async job submit for node 69c21ddb0efaf21d93107741 2026-03-26 05:15:24,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:15:24,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21ddb0efaf21d93107741 HTTP/1.1" 200 None 2026-03-26 05:15:24,267 - lava_callback - DEBUG - Processing node 69c21ddb0efaf21d93107741: baseline-arm64-baylibre 2026-03-26 05:15:24,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21ddb0efaf21d93107741 2026-03-26 05:15:24,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:15:24,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21ddb0efaf21d93107741 HTTP/1.1" 200 None 2026-03-26 05:15:24,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:15:24,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:15:24,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:15:24,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21ddb0efaf21d93107741 HTTP/1.1" 200 None 2026-03-26 05:15:24,344 - lava_callback - ERROR - Error processing callback for node 69c21ddb0efaf21d93107741: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41124 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:16:44,310 - lava_callback - INFO - Received callback for node 69c21e310efaf21d9310774f INFO: 172.21.0.1:34748 - "POST /node/69c21e310efaf21d9310774f HTTP/1.0" 202 Accepted 2026-03-26 05:16:44,316 - lava_callback - INFO - Starting async job submit for node 69c21e310efaf21d9310774f 2026-03-26 05:16:44,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:16:44,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21e310efaf21d9310774f HTTP/1.1" 200 None 2026-03-26 05:16:44,335 - lava_callback - DEBUG - Processing node 69c21e310efaf21d9310774f: baseline-arm64-baylibre 2026-03-26 05:16:44,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21e310efaf21d9310774f 2026-03-26 05:16:44,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:16:44,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21e310efaf21d9310774f HTTP/1.1" 200 None 2026-03-26 05:16:44,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:16:44,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:16:44,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:16:44,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21e310efaf21d9310774f HTTP/1.1" 200 None 2026-03-26 05:16:44,395 - lava_callback - ERROR - Error processing callback for node 69c21e310efaf21d9310774f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56264 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:18:30,504 - lava_callback - INFO - Received callback for node 69c21e920efaf21d9310775e INFO: 172.21.0.1:33626 - "POST /node/69c21e920efaf21d9310775e HTTP/1.0" 202 Accepted 2026-03-26 05:18:30,509 - lava_callback - INFO - Starting async job submit for node 69c21e920efaf21d9310775e 2026-03-26 05:18:30,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:18:30,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21e920efaf21d9310775e HTTP/1.1" 200 None 2026-03-26 05:18:30,526 - lava_callback - DEBUG - Processing node 69c21e920efaf21d9310775e: baseline-arm64-clabbe 2026-03-26 05:18:30,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21e920efaf21d9310775e 2026-03-26 05:18:30,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:18:30,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21e920efaf21d9310775e HTTP/1.1" 200 None 2026-03-26 05:18:30,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:18:30,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:18:30,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:18:30,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21e920efaf21d9310775e HTTP/1.1" 200 None 2026-03-26 05:18:30,611 - lava_callback - ERROR - Error processing callback for node 69c21e920efaf21d9310775e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35660 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:19:44,525 - lava_callback - INFO - Received callback for node 69c21ee20efaf21d9310776b INFO: 172.21.0.1:35264 - "POST /node/69c21ee20efaf21d9310776b HTTP/1.0" 202 Accepted 2026-03-26 05:19:44,530 - lava_callback - INFO - Starting async job submit for node 69c21ee20efaf21d9310776b 2026-03-26 05:19:44,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:19:44,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21ee20efaf21d9310776b HTTP/1.1" 200 None 2026-03-26 05:19:44,546 - lava_callback - DEBUG - Processing node 69c21ee20efaf21d9310776b: baseline-arm64-baylibre 2026-03-26 05:19:44,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21ee20efaf21d9310776b 2026-03-26 05:19:44,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:19:44,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21ee20efaf21d9310776b HTTP/1.1" 200 None 2026-03-26 05:19:44,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:19:44,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:19:44,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:19:44,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21ee20efaf21d9310776b HTTP/1.1" 200 None 2026-03-26 05:19:44,599 - lava_callback - ERROR - Error processing callback for node 69c21ee20efaf21d9310776b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51286 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:21:10,534 - lava_callback - INFO - Received callback for node 69c21f410efaf21d9310777f INFO: 172.21.0.1:53460 - "POST /node/69c21f410efaf21d9310777f HTTP/1.0" 202 Accepted 2026-03-26 05:21:10,540 - lava_callback - INFO - Starting async job submit for node 69c21f410efaf21d9310777f 2026-03-26 05:21:10,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:21:10,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21f410efaf21d9310777f HTTP/1.1" 200 None 2026-03-26 05:21:10,556 - lava_callback - DEBUG - Processing node 69c21f410efaf21d9310777f: baseline-arm64-clabbe 2026-03-26 05:21:10,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21f410efaf21d9310777f 2026-03-26 05:21:10,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:21:10,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21f410efaf21d9310777f HTTP/1.1" 200 None 2026-03-26 05:21:10,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:21:10,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:21:10,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:21:10,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21f410efaf21d9310777f HTTP/1.1" 200 None 2026-03-26 05:21:10,654 - lava_callback - ERROR - Error processing callback for node 69c21f410efaf21d9310777f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43910 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:22:44,646 - lava_callback - INFO - Received callback for node 69c21f910efaf21d93107796 INFO: 172.21.0.1:35646 - "POST /node/69c21f910efaf21d93107796 HTTP/1.0" 202 Accepted 2026-03-26 05:22:44,651 - lava_callback - INFO - Starting async job submit for node 69c21f910efaf21d93107796 2026-03-26 05:22:44,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:22:44,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21f910efaf21d93107796 HTTP/1.1" 200 None 2026-03-26 05:22:44,666 - lava_callback - DEBUG - Processing node 69c21f910efaf21d93107796: baseline-arm64-baylibre 2026-03-26 05:22:44,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21f910efaf21d93107796 2026-03-26 05:22:44,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:22:44,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21f910efaf21d93107796 HTTP/1.1" 200 None 2026-03-26 05:22:44,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:22:44,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:22:44,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:22:44,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21f910efaf21d93107796 HTTP/1.1" 200 None 2026-03-26 05:22:44,732 - lava_callback - ERROR - Error processing callback for node 69c21f910efaf21d93107796: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40294 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:24:04,767 - lava_callback - INFO - Received callback for node 69c21fe10efaf21d931077b2 INFO: 172.21.0.1:42250 - "POST /node/69c21fe10efaf21d931077b2 HTTP/1.0" 202 Accepted 2026-03-26 05:24:04,772 - lava_callback - INFO - Starting async job submit for node 69c21fe10efaf21d931077b2 2026-03-26 05:24:04,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:24:04,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21fe10efaf21d931077b2 HTTP/1.1" 200 None 2026-03-26 05:24:04,795 - lava_callback - DEBUG - Processing node 69c21fe10efaf21d931077b2: baseline-arm64-baylibre 2026-03-26 05:24:04,796 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c21fe10efaf21d931077b2 2026-03-26 05:24:04,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:24:04,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c21fe10efaf21d931077b2 HTTP/1.1" 200 None 2026-03-26 05:24:04,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:24:04,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:24:04,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:24:04,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c21fe10efaf21d931077b2 HTTP/1.1" 200 None 2026-03-26 05:24:04,852 - lava_callback - ERROR - Error processing callback for node 69c21fe10efaf21d931077b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45318 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:25:30,686 - lava_callback - INFO - Received callback for node 69c2203e0efaf21d931077cc INFO: 172.21.0.1:57566 - "POST /node/69c2203e0efaf21d931077cc HTTP/1.0" 202 Accepted 2026-03-26 05:25:30,692 - lava_callback - INFO - Starting async job submit for node 69c2203e0efaf21d931077cc 2026-03-26 05:25:30,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:25:30,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2203e0efaf21d931077cc HTTP/1.1" 200 None 2026-03-26 05:25:30,709 - lava_callback - DEBUG - Processing node 69c2203e0efaf21d931077cc: baseline-arm64-clabbe 2026-03-26 05:25:30,709 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2203e0efaf21d931077cc 2026-03-26 05:25:30,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:25:30,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2203e0efaf21d931077cc HTTP/1.1" 200 None 2026-03-26 05:25:30,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:25:30,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:25:30,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:25:30,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2203e0efaf21d931077cc HTTP/1.1" 200 None 2026-03-26 05:25:30,969 - lava_callback - ERROR - Error processing callback for node 69c2203e0efaf21d931077cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53474 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:26:44,948 - lava_callback - INFO - Received callback for node 69c220900efaf21d931077e4 INFO: 172.21.0.1:60746 - "POST /node/69c220900efaf21d931077e4 HTTP/1.0" 202 Accepted 2026-03-26 05:26:44,954 - lava_callback - INFO - Starting async job submit for node 69c220900efaf21d931077e4 2026-03-26 05:26:44,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:26:44,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c220900efaf21d931077e4 HTTP/1.1" 200 None 2026-03-26 05:26:44,981 - lava_callback - DEBUG - Processing node 69c220900efaf21d931077e4: baseline-arm64-baylibre 2026-03-26 05:26:44,981 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c220900efaf21d931077e4 2026-03-26 05:26:44,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:26:44,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c220900efaf21d931077e4 HTTP/1.1" 200 None 2026-03-26 05:26:44,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:26:45,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:26:45,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:26:45,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c220900efaf21d931077e4 HTTP/1.1" 200 None 2026-03-26 05:26:45,036 - lava_callback - ERROR - Error processing callback for node 69c220900efaf21d931077e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:28:30,787 - lava_callback - INFO - Received callback for node 69c220ef0efaf21d931077f9 INFO: 172.21.0.1:44738 - "POST /node/69c220ef0efaf21d931077f9 HTTP/1.0" 202 Accepted 2026-03-26 05:28:30,793 - lava_callback - INFO - Starting async job submit for node 69c220ef0efaf21d931077f9 2026-03-26 05:28:30,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:28:30,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c220ef0efaf21d931077f9 HTTP/1.1" 200 None 2026-03-26 05:28:30,809 - lava_callback - DEBUG - Processing node 69c220ef0efaf21d931077f9: baseline-arm64-clabbe 2026-03-26 05:28:30,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c220ef0efaf21d931077f9 2026-03-26 05:28:30,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:28:30,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c220ef0efaf21d931077f9 HTTP/1.1" 200 None 2026-03-26 05:28:30,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:28:30,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:28:30,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:28:30,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c220ef0efaf21d931077f9 HTTP/1.1" 200 None 2026-03-26 05:28:30,862 - lava_callback - ERROR - Error processing callback for node 69c220ef0efaf21d931077f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33714 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:29:50,913 - lava_callback - INFO - Received callback for node 69c221440efaf21d93107810 INFO: 172.21.0.1:45156 - "POST /node/69c221440efaf21d93107810 HTTP/1.0" 202 Accepted 2026-03-26 05:29:50,918 - lava_callback - INFO - Starting async job submit for node 69c221440efaf21d93107810 2026-03-26 05:29:50,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:29:50,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c221440efaf21d93107810 HTTP/1.1" 200 None 2026-03-26 05:29:50,935 - lava_callback - DEBUG - Processing node 69c221440efaf21d93107810: baseline-arm64-clabbe 2026-03-26 05:29:50,935 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c221440efaf21d93107810 2026-03-26 05:29:50,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:29:50,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c221440efaf21d93107810 HTTP/1.1" 200 None 2026-03-26 05:29:50,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:29:50,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:29:50,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:29:50,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c221440efaf21d93107810 HTTP/1.1" 200 None 2026-03-26 05:29:50,991 - lava_callback - ERROR - Error processing callback for node 69c221440efaf21d93107810: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37430 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:31:05,246 - lava_callback - INFO - Received callback for node 69c221900efaf21d93107823 INFO: 172.21.0.1:48054 - "POST /node/69c221900efaf21d93107823 HTTP/1.0" 202 Accepted 2026-03-26 05:31:05,252 - lava_callback - INFO - Starting async job submit for node 69c221900efaf21d93107823 2026-03-26 05:31:05,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:31:05,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c221900efaf21d93107823 HTTP/1.1" 200 None 2026-03-26 05:31:05,268 - lava_callback - DEBUG - Processing node 69c221900efaf21d93107823: baseline-arm64-baylibre 2026-03-26 05:31:05,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c221900efaf21d93107823 2026-03-26 05:31:05,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:31:05,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c221900efaf21d93107823 HTTP/1.1" 200 None 2026-03-26 05:31:05,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:31:05,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:31:05,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:31:05,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c221900efaf21d93107823 HTTP/1.1" 200 None 2026-03-26 05:31:05,323 - lava_callback - ERROR - Error processing callback for node 69c221900efaf21d93107823: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35204 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:32:51,020 - lava_callback - INFO - Received callback for node 69c221ed0efaf21d93107838 INFO: 172.21.0.1:49130 - "POST /node/69c221ed0efaf21d93107838 HTTP/1.0" 202 Accepted 2026-03-26 05:32:51,026 - lava_callback - INFO - Starting async job submit for node 69c221ed0efaf21d93107838 2026-03-26 05:32:51,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:32:51,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c221ed0efaf21d93107838 HTTP/1.1" 200 None 2026-03-26 05:32:51,042 - lava_callback - DEBUG - Processing node 69c221ed0efaf21d93107838: baseline-arm64-clabbe 2026-03-26 05:32:51,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c221ed0efaf21d93107838 2026-03-26 05:32:51,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:32:51,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c221ed0efaf21d93107838 HTTP/1.1" 200 None 2026-03-26 05:32:51,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:32:51,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:32:51,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:32:51,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c221ed0efaf21d93107838 HTTP/1.1" 200 None 2026-03-26 05:32:51,097 - lava_callback - ERROR - Error processing callback for node 69c221ed0efaf21d93107838: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59028 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:33:45,323 - lava_callback - INFO - Received callback for node 69c222350efaf21d9310784f INFO: 172.21.0.1:55166 - "POST /node/69c222350efaf21d9310784f HTTP/1.0" 202 Accepted 2026-03-26 05:33:45,329 - lava_callback - INFO - Starting async job submit for node 69c222350efaf21d9310784f 2026-03-26 05:33:45,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:33:45,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c222350efaf21d9310784f HTTP/1.1" 200 None 2026-03-26 05:33:45,370 - lava_callback - DEBUG - Processing node 69c222350efaf21d9310784f: baseline-arm64-baylibre 2026-03-26 05:33:45,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c222350efaf21d9310784f 2026-03-26 05:33:45,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:33:45,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c222350efaf21d9310784f HTTP/1.1" 200 None 2026-03-26 05:33:45,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:33:45,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:33:45,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:33:45,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c222350efaf21d9310784f HTTP/1.1" 200 None 2026-03-26 05:33:45,426 - lava_callback - ERROR - Error processing callback for node 69c222350efaf21d9310784f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:35:11,147 - lava_callback - INFO - Received callback for node 69c2228b0efaf21d93107868 INFO: 172.21.0.1:57198 - "POST /node/69c2228b0efaf21d93107868 HTTP/1.0" 202 Accepted 2026-03-26 05:35:11,153 - lava_callback - INFO - Starting async job submit for node 69c2228b0efaf21d93107868 2026-03-26 05:35:11,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:35:11,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2228b0efaf21d93107868 HTTP/1.1" 200 None 2026-03-26 05:35:11,169 - lava_callback - DEBUG - Processing node 69c2228b0efaf21d93107868: baseline-arm64-clabbe 2026-03-26 05:35:11,169 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2228b0efaf21d93107868 2026-03-26 05:35:11,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:35:11,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2228b0efaf21d93107868 HTTP/1.1" 200 None 2026-03-26 05:35:11,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:35:11,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:35:11,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:35:11,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2228b0efaf21d93107868 HTTP/1.1" 200 None 2026-03-26 05:35:11,235 - lava_callback - ERROR - Error processing callback for node 69c2228b0efaf21d93107868: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:36:25,421 - lava_callback - INFO - Received callback for node 69c222cf0efaf21d9310787f INFO: 172.21.0.1:44240 - "POST /node/69c222cf0efaf21d9310787f HTTP/1.0" 202 Accepted 2026-03-26 05:36:25,426 - lava_callback - INFO - Starting async job submit for node 69c222cf0efaf21d9310787f 2026-03-26 05:36:25,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:36:25,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c222cf0efaf21d9310787f HTTP/1.1" 200 None 2026-03-26 05:36:25,466 - lava_callback - DEBUG - Processing node 69c222cf0efaf21d9310787f: baseline-arm-baylibre 2026-03-26 05:36:25,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c222cf0efaf21d9310787f 2026-03-26 05:36:25,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:36:25,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c222cf0efaf21d9310787f HTTP/1.1" 200 None 2026-03-26 05:36:25,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:36:25,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:36:25,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:36:25,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c222cf0efaf21d9310787f HTTP/1.1" 200 None 2026-03-26 05:36:25,526 - lava_callback - ERROR - Error processing callback for node 69c222cf0efaf21d9310787f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55088 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:37:05,386 - lava_callback - INFO - Received callback for node 69c222f60efaf21d9310788f INFO: 172.21.0.1:47948 - "POST /node/69c222f60efaf21d9310788f HTTP/1.0" 202 Accepted 2026-03-26 05:37:05,392 - lava_callback - INFO - Starting async job submit for node 69c222f60efaf21d9310788f 2026-03-26 05:37:05,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:37:05,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c222f60efaf21d9310788f HTTP/1.1" 200 None 2026-03-26 05:37:05,410 - lava_callback - DEBUG - Processing node 69c222f60efaf21d9310788f: baseline-arm-baylibre 2026-03-26 05:37:05,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c222f60efaf21d9310788f 2026-03-26 05:37:05,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:37:05,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c222f60efaf21d9310788f HTTP/1.1" 200 None 2026-03-26 05:37:05,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:37:05,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:37:05,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:37:05,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c222f60efaf21d9310788f HTTP/1.1" 200 None 2026-03-26 05:37:05,477 - lava_callback - ERROR - Error processing callback for node 69c222f60efaf21d9310788f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37394 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:38:11,237 - lava_callback - INFO - Received callback for node 69c2232f0efaf21d931078a3 INFO: 172.21.0.1:60498 - "POST /node/69c2232f0efaf21d931078a3 HTTP/1.0" 202 Accepted 2026-03-26 05:38:11,242 - lava_callback - INFO - Starting async job submit for node 69c2232f0efaf21d931078a3 2026-03-26 05:38:11,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:38:11,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2232f0efaf21d931078a3 HTTP/1.1" 200 None 2026-03-26 05:38:11,259 - lava_callback - DEBUG - Processing node 69c2232f0efaf21d931078a3: baseline-arm64-clabbe 2026-03-26 05:38:11,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2232f0efaf21d931078a3 2026-03-26 05:38:11,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:38:11,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2232f0efaf21d931078a3 HTTP/1.1" 200 None 2026-03-26 05:38:11,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:38:11,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:38:11,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:38:11,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2232f0efaf21d931078a3 HTTP/1.1" 200 None 2026-03-26 05:38:11,323 - lava_callback - ERROR - Error processing callback for node 69c2232f0efaf21d931078a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51580 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:39:05,531 - lava_callback - INFO - Received callback for node 69c223730efaf21d931078b7 INFO: 172.21.0.1:49012 - "POST /node/69c223730efaf21d931078b7 HTTP/1.0" 202 Accepted 2026-03-26 05:39:05,537 - lava_callback - INFO - Starting async job submit for node 69c223730efaf21d931078b7 2026-03-26 05:39:05,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:39:05,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c223730efaf21d931078b7 HTTP/1.1" 200 None 2026-03-26 05:39:05,553 - lava_callback - DEBUG - Processing node 69c223730efaf21d931078b7: baseline-arm-baylibre 2026-03-26 05:39:05,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c223730efaf21d931078b7 2026-03-26 05:39:05,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:39:05,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c223730efaf21d931078b7 HTTP/1.1" 200 None 2026-03-26 05:39:05,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:39:05,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:39:05,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:39:05,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c223730efaf21d931078b7 HTTP/1.1" 200 None 2026-03-26 05:39:05,639 - lava_callback - ERROR - Error processing callback for node 69c223730efaf21d931078b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40832 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:40:11,315 - lava_callback - INFO - Received callback for node 69c223af0efaf21d931078ce INFO: 172.21.0.1:39720 - "POST /node/69c223af0efaf21d931078ce HTTP/1.0" 202 Accepted 2026-03-26 05:40:11,321 - lava_callback - INFO - Starting async job submit for node 69c223af0efaf21d931078ce 2026-03-26 05:40:11,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:40:11,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c223af0efaf21d931078ce HTTP/1.1" 200 None 2026-03-26 05:40:11,340 - lava_callback - DEBUG - Processing node 69c223af0efaf21d931078ce: baseline-arm64-clabbe 2026-03-26 05:40:11,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c223af0efaf21d931078ce 2026-03-26 05:40:11,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:40:11,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c223af0efaf21d931078ce HTTP/1.1" 200 None 2026-03-26 05:40:11,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:40:11,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:40:11,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:40:11,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c223af0efaf21d931078ce HTTP/1.1" 200 None 2026-03-26 05:40:11,596 - lava_callback - ERROR - Error processing callback for node 69c223af0efaf21d931078ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:41:31,390 - lava_callback - INFO - Received callback for node 69c224070efaf21d931078e7 INFO: 172.21.0.1:48530 - "POST /node/69c224070efaf21d931078e7 HTTP/1.0" 202 Accepted 2026-03-26 05:41:31,395 - lava_callback - INFO - Starting async job submit for node 69c224070efaf21d931078e7 2026-03-26 05:41:31,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:41:31,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224070efaf21d931078e7 HTTP/1.1" 200 None 2026-03-26 05:41:31,412 - lava_callback - DEBUG - Processing node 69c224070efaf21d931078e7: baseline-arm64-clabbe 2026-03-26 05:41:31,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c224070efaf21d931078e7 2026-03-26 05:41:31,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:41:31,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224070efaf21d931078e7 HTTP/1.1" 200 None 2026-03-26 05:41:31,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:41:31,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:41:31,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:41:31,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c224070efaf21d931078e7 HTTP/1.1" 200 None 2026-03-26 05:41:31,465 - lava_callback - ERROR - Error processing callback for node 69c224070efaf21d931078e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49356 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:42:51,448 - lava_callback - INFO - Received callback for node 69c2244e0efaf21d93107909 INFO: 172.21.0.1:44556 - "POST /node/69c2244e0efaf21d93107909 HTTP/1.0" 202 Accepted 2026-03-26 05:42:51,453 - lava_callback - INFO - Starting async job submit for node 69c2244e0efaf21d93107909 2026-03-26 05:42:51,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:42:51,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2244e0efaf21d93107909 HTTP/1.1" 200 None 2026-03-26 05:42:51,477 - lava_callback - DEBUG - Processing node 69c2244e0efaf21d93107909: baseline-arm-clabbe 2026-03-26 05:42:51,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2244e0efaf21d93107909 2026-03-26 05:42:51,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:42:51,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2244e0efaf21d93107909 HTTP/1.1" 200 None 2026-03-26 05:42:51,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:42:51,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:42:51,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:42:51,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2244e0efaf21d93107909 HTTP/1.1" 200 None 2026-03-26 05:42:51,607 - lava_callback - ERROR - Error processing callback for node 69c2244e0efaf21d93107909: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40896 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:43:31,439 - lava_callback - INFO - Received callback for node 69c224760efaf21d9310791b INFO: 172.21.0.1:44292 - "POST /node/69c224760efaf21d9310791b HTTP/1.0" 202 Accepted 2026-03-26 05:43:31,444 - lava_callback - INFO - Starting async job submit for node 69c224760efaf21d9310791b 2026-03-26 05:43:31,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:43:31,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224760efaf21d9310791b HTTP/1.1" 200 None 2026-03-26 05:43:31,466 - lava_callback - DEBUG - Processing node 69c224760efaf21d9310791b: baseline-arm-clabbe 2026-03-26 05:43:31,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c224760efaf21d9310791b 2026-03-26 05:43:31,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:43:31,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224760efaf21d9310791b HTTP/1.1" 200 None 2026-03-26 05:43:31,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:43:31,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:43:31,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:43:31,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c224760efaf21d9310791b HTTP/1.1" 200 None 2026-03-26 05:43:31,543 - lava_callback - ERROR - Error processing callback for node 69c224760efaf21d9310791b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57134 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:44:31,515 - lava_callback - INFO - Received callback for node 69c224ac0efaf21d9310792f INFO: 172.21.0.1:48352 - "POST /node/69c224ac0efaf21d9310792f HTTP/1.0" 202 Accepted 2026-03-26 05:44:31,521 - lava_callback - INFO - Starting async job submit for node 69c224ac0efaf21d9310792f 2026-03-26 05:44:31,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:44:31,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224ac0efaf21d9310792f HTTP/1.1" 200 None 2026-03-26 05:44:31,537 - lava_callback - DEBUG - Processing node 69c224ac0efaf21d9310792f: baseline-arm64-clabbe 2026-03-26 05:44:31,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c224ac0efaf21d9310792f 2026-03-26 05:44:31,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:44:31,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224ac0efaf21d9310792f HTTP/1.1" 200 None 2026-03-26 05:44:31,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:44:31,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:44:31,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:44:31,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c224ac0efaf21d9310792f HTTP/1.1" 200 None 2026-03-26 05:44:31,616 - lava_callback - ERROR - Error processing callback for node 69c224ac0efaf21d9310792f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44766 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:45:31,622 - lava_callback - INFO - Received callback for node 69c224f20efaf21d93107945 INFO: 172.21.0.1:38676 - "POST /node/69c224f20efaf21d93107945 HTTP/1.0" 202 Accepted 2026-03-26 05:45:31,628 - lava_callback - INFO - Starting async job submit for node 69c224f20efaf21d93107945 2026-03-26 05:45:31,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:45:31,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224f20efaf21d93107945 HTTP/1.1" 200 None 2026-03-26 05:45:31,684 - lava_callback - DEBUG - Processing node 69c224f20efaf21d93107945: baseline-arm-clabbe 2026-03-26 05:45:31,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c224f20efaf21d93107945 2026-03-26 05:45:31,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:45:31,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c224f20efaf21d93107945 HTTP/1.1" 200 None 2026-03-26 05:45:31,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:45:31,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:45:31,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:45:31,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c224f20efaf21d93107945 HTTP/1.1" 200 None 2026-03-26 05:45:31,767 - lava_callback - ERROR - Error processing callback for node 69c224f20efaf21d93107945: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52932 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:46:11,545 - lava_callback - INFO - Received callback for node 69c225190efaf21d9310795b INFO: 172.21.0.1:43434 - "POST /node/69c225190efaf21d9310795b HTTP/1.0" 202 Accepted 2026-03-26 05:46:11,551 - lava_callback - INFO - Starting async job submit for node 69c225190efaf21d9310795b 2026-03-26 05:46:11,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:11,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225190efaf21d9310795b HTTP/1.1" 200 None 2026-03-26 05:46:11,567 - lava_callback - DEBUG - Processing node 69c225190efaf21d9310795b: baseline-arm-clabbe 2026-03-26 05:46:11,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c225190efaf21d9310795b 2026-03-26 05:46:11,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:11,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225190efaf21d9310795b HTTP/1.1" 200 None 2026-03-26 05:46:11,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:11,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:46:11,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:11,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c225190efaf21d9310795b HTTP/1.1" 200 None 2026-03-26 05:46:11,629 - lava_callback - ERROR - Error processing callback for node 69c225190efaf21d9310795b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:46:51,636 - lava_callback - INFO - Received callback for node 69c225400efaf21d93107968 INFO: 172.21.0.1:49994 - "POST /node/69c225400efaf21d93107968 HTTP/1.0" 202 Accepted 2026-03-26 05:46:51,641 - lava_callback - INFO - Starting async job submit for node 69c225400efaf21d93107968 2026-03-26 05:46:51,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:51,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225400efaf21d93107968 HTTP/1.1" 200 None 2026-03-26 05:46:51,659 - lava_callback - DEBUG - Processing node 69c225400efaf21d93107968: baseline-arm-clabbe 2026-03-26 05:46:51,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c225400efaf21d93107968 2026-03-26 05:46:51,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:51,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225400efaf21d93107968 HTTP/1.1" 200 None 2026-03-26 05:46:51,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:51,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:46:51,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:46:51,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c225400efaf21d93107968 HTTP/1.1" 200 None 2026-03-26 05:46:51,737 - lava_callback - ERROR - Error processing callback for node 69c225400efaf21d93107968: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:47:51,672 - lava_callback - INFO - Received callback for node 69c225760efaf21d9310797a INFO: 172.21.0.1:34410 - "POST /node/69c225760efaf21d9310797a HTTP/1.0" 202 Accepted 2026-03-26 05:47:51,677 - lava_callback - INFO - Starting async job submit for node 69c225760efaf21d9310797a 2026-03-26 05:47:51,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:47:51,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225760efaf21d9310797a HTTP/1.1" 200 None 2026-03-26 05:47:51,719 - lava_callback - DEBUG - Processing node 69c225760efaf21d9310797a: baseline-arm64-clabbe 2026-03-26 05:47:51,719 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c225760efaf21d9310797a 2026-03-26 05:47:51,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:47:51,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225760efaf21d9310797a HTTP/1.1" 200 None 2026-03-26 05:47:51,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:47:51,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:47:51,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:47:51,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c225760efaf21d9310797a HTTP/1.1" 200 None 2026-03-26 05:47:51,817 - lava_callback - ERROR - Error processing callback for node 69c225760efaf21d9310797a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33098 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:49:11,757 - lava_callback - INFO - Received callback for node 69c225cc0efaf21d9310798c INFO: 172.21.0.1:55520 - "POST /node/69c225cc0efaf21d9310798c HTTP/1.0" 202 Accepted 2026-03-26 05:49:11,763 - lava_callback - INFO - Starting async job submit for node 69c225cc0efaf21d9310798c 2026-03-26 05:49:11,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:49:11,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225cc0efaf21d9310798c HTTP/1.1" 200 None 2026-03-26 05:49:11,783 - lava_callback - DEBUG - Processing node 69c225cc0efaf21d9310798c: baseline-arm64-clabbe 2026-03-26 05:49:11,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c225cc0efaf21d9310798c 2026-03-26 05:49:11,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:49:11,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c225cc0efaf21d9310798c HTTP/1.1" 200 None 2026-03-26 05:49:11,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:49:11,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:49:11,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:49:11,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c225cc0efaf21d9310798c HTTP/1.1" 200 None 2026-03-26 05:49:11,842 - lava_callback - ERROR - Error processing callback for node 69c225cc0efaf21d9310798c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:50:31,738 - lava_callback - INFO - Received callback for node 69c226250efaf21d9310799f INFO: 172.21.0.1:44220 - "POST /node/69c226250efaf21d9310799f HTTP/1.0" 202 Accepted 2026-03-26 05:50:31,744 - lava_callback - INFO - Starting async job submit for node 69c226250efaf21d9310799f 2026-03-26 05:50:31,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:50:31,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c226250efaf21d9310799f HTTP/1.1" 200 None 2026-03-26 05:50:31,759 - lava_callback - DEBUG - Processing node 69c226250efaf21d9310799f: baseline-arm64-clabbe 2026-03-26 05:50:31,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c226250efaf21d9310799f 2026-03-26 05:50:31,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:50:31,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c226250efaf21d9310799f HTTP/1.1" 200 None 2026-03-26 05:50:31,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:50:31,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:50:31,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:50:31,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c226250efaf21d9310799f HTTP/1.1" 200 None 2026-03-26 05:50:31,820 - lava_callback - ERROR - Error processing callback for node 69c226250efaf21d9310799f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:52:11,887 - lava_callback - INFO - Received callback for node 69c2267c0efaf21d931079b5 INFO: 172.21.0.1:51336 - "POST /node/69c2267c0efaf21d931079b5 HTTP/1.0" 202 Accepted 2026-03-26 05:52:11,893 - lava_callback - INFO - Starting async job submit for node 69c2267c0efaf21d931079b5 2026-03-26 05:52:11,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:52:11,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2267c0efaf21d931079b5 HTTP/1.1" 200 None 2026-03-26 05:52:11,919 - lava_callback - DEBUG - Processing node 69c2267c0efaf21d931079b5: baseline-arm64-clabbe 2026-03-26 05:52:11,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2267c0efaf21d931079b5 2026-03-26 05:52:11,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:52:11,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2267c0efaf21d931079b5 HTTP/1.1" 200 None 2026-03-26 05:52:11,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:52:11,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:52:11,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:52:11,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2267c0efaf21d931079b5 HTTP/1.1" 200 None 2026-03-26 05:52:11,988 - lava_callback - ERROR - Error processing callback for node 69c2267c0efaf21d931079b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57376 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:53:31,842 - lava_callback - INFO - Received callback for node 69c226d20efaf21d931079c6 INFO: 172.21.0.1:50106 - "POST /node/69c226d20efaf21d931079c6 HTTP/1.0" 202 Accepted 2026-03-26 05:53:31,847 - lava_callback - INFO - Starting async job submit for node 69c226d20efaf21d931079c6 2026-03-26 05:53:31,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:53:31,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c226d20efaf21d931079c6 HTTP/1.1" 200 None 2026-03-26 05:53:31,863 - lava_callback - DEBUG - Processing node 69c226d20efaf21d931079c6: baseline-arm64-clabbe 2026-03-26 05:53:31,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c226d20efaf21d931079c6 2026-03-26 05:53:31,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:53:31,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c226d20efaf21d931079c6 HTTP/1.1" 200 None 2026-03-26 05:53:31,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:53:31,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:53:31,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:53:31,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c226d20efaf21d931079c6 HTTP/1.1" 200 None 2026-03-26 05:53:31,939 - lava_callback - ERROR - Error processing callback for node 69c226d20efaf21d931079c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55608 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:54:51,907 - lava_callback - INFO - Received callback for node 69c227290efaf21d931079de INFO: 172.21.0.1:53740 - "POST /node/69c227290efaf21d931079de HTTP/1.0" 202 Accepted 2026-03-26 05:54:51,912 - lava_callback - INFO - Starting async job submit for node 69c227290efaf21d931079de 2026-03-26 05:54:51,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:54:51,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c227290efaf21d931079de HTTP/1.1" 200 None 2026-03-26 05:54:51,952 - lava_callback - DEBUG - Processing node 69c227290efaf21d931079de: baseline-arm64-clabbe 2026-03-26 05:54:51,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c227290efaf21d931079de 2026-03-26 05:54:51,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:54:51,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c227290efaf21d931079de HTTP/1.1" 200 None 2026-03-26 05:54:51,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:54:51,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:54:51,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:54:52,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c227290efaf21d931079de HTTP/1.1" 200 None 2026-03-26 05:54:52,033 - lava_callback - ERROR - Error processing callback for node 69c227290efaf21d931079de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34996 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:56:06,609 - lava_callback - INFO - Received callback for node 69c2276e0efaf21d931079f4 INFO: 172.21.0.1:35002 - "POST /node/69c2276e0efaf21d931079f4 HTTP/1.0" 202 Accepted 2026-03-26 05:56:06,615 - lava_callback - INFO - Starting async job submit for node 69c2276e0efaf21d931079f4 2026-03-26 05:56:06,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:56:06,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2276e0efaf21d931079f4 HTTP/1.1" 200 None 2026-03-26 05:56:06,632 - lava_callback - DEBUG - Processing node 69c2276e0efaf21d931079f4: baseline-arm-baylibre 2026-03-26 05:56:06,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2276e0efaf21d931079f4 2026-03-26 05:56:06,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:56:06,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2276e0efaf21d931079f4 HTTP/1.1" 200 None 2026-03-26 05:56:06,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:56:06,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 05:56:06,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:56:06,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2276e0efaf21d931079f4 HTTP/1.1" 200 None 2026-03-26 05:56:06,693 - lava_callback - ERROR - Error processing callback for node 69c2276e0efaf21d931079f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38496 - "GET / HTTP/1.0" 200 OK 2026-03-26 05:58:32,088 - lava_callback - INFO - Received callback for node 69c227f30efaf21d93107a24 INFO: 172.21.0.1:34872 - "POST /node/69c227f30efaf21d93107a24 HTTP/1.0" 202 Accepted 2026-03-26 05:58:32,093 - lava_callback - INFO - Starting async job submit for node 69c227f30efaf21d93107a24 2026-03-26 05:58:32,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:58:32,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c227f30efaf21d93107a24 HTTP/1.1" 200 None 2026-03-26 05:58:32,109 - lava_callback - DEBUG - Processing node 69c227f30efaf21d93107a24: baseline-arm64-clabbe 2026-03-26 05:58:32,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c227f30efaf21d93107a24 2026-03-26 05:58:32,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:58:32,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c227f30efaf21d93107a24 HTTP/1.1" 200 None 2026-03-26 05:58:32,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:58:32,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-26 05:58:32,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 05:58:32,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c227f30efaf21d93107a24 HTTP/1.1" 200 None 2026-03-26 05:58:32,347 - lava_callback - ERROR - Error processing callback for node 69c227f30efaf21d93107a24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59630 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:01:32,232 - lava_callback - INFO - Received callback for node 69c228a80efaf21d93107a5e INFO: 172.21.0.1:48358 - "POST /node/69c228a80efaf21d93107a5e HTTP/1.0" 202 Accepted 2026-03-26 06:01:32,238 - lava_callback - INFO - Starting async job submit for node 69c228a80efaf21d93107a5e 2026-03-26 06:01:32,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:01:32,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c228a80efaf21d93107a5e HTTP/1.1" 200 None 2026-03-26 06:01:32,256 - lava_callback - DEBUG - Processing node 69c228a80efaf21d93107a5e: baseline-arm-clabbe 2026-03-26 06:01:32,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c228a80efaf21d93107a5e 2026-03-26 06:01:32,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:01:32,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c228a80efaf21d93107a5e HTTP/1.1" 200 None 2026-03-26 06:01:32,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:01:32,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:01:32,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:01:32,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c228a80efaf21d93107a5e HTTP/1.1" 200 None 2026-03-26 06:01:32,352 - lava_callback - ERROR - Error processing callback for node 69c228a80efaf21d93107a5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34066 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:02:32,356 - lava_callback - INFO - Received callback for node 69c228f50efaf21d93107a81 INFO: 172.21.0.1:50054 - "POST /node/69c228f50efaf21d93107a81 HTTP/1.0" 202 Accepted 2026-03-26 06:02:32,361 - lava_callback - INFO - Starting async job submit for node 69c228f50efaf21d93107a81 2026-03-26 06:02:32,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:02:32,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c228f50efaf21d93107a81 HTTP/1.1" 200 None 2026-03-26 06:02:32,377 - lava_callback - DEBUG - Processing node 69c228f50efaf21d93107a81: baseline-arm-clabbe 2026-03-26 06:02:32,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c228f50efaf21d93107a81 2026-03-26 06:02:32,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:02:32,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c228f50efaf21d93107a81 HTTP/1.1" 200 None 2026-03-26 06:02:32,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:02:32,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:02:32,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:02:32,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c228f50efaf21d93107a81 HTTP/1.1" 200 None 2026-03-26 06:02:32,443 - lava_callback - ERROR - Error processing callback for node 69c228f50efaf21d93107a81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33668 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:04:32,500 - lava_callback - INFO - Received callback for node 69c229650efaf21d93107aa8 INFO: 172.21.0.1:42554 - "POST /node/69c229650efaf21d93107aa8 HTTP/1.0" 202 Accepted 2026-03-26 06:04:32,506 - lava_callback - INFO - Starting async job submit for node 69c229650efaf21d93107aa8 2026-03-26 06:04:32,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:04:32,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229650efaf21d93107aa8 HTTP/1.1" 200 None 2026-03-26 06:04:32,534 - lava_callback - DEBUG - Processing node 69c229650efaf21d93107aa8: baseline-arm-clabbe 2026-03-26 06:04:32,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c229650efaf21d93107aa8 2026-03-26 06:04:32,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:04:32,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229650efaf21d93107aa8 HTTP/1.1" 200 None 2026-03-26 06:04:32,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:04:32,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:04:32,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:04:32,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c229650efaf21d93107aa8 HTTP/1.1" 200 None 2026-03-26 06:04:32,608 - lava_callback - ERROR - Error processing callback for node 69c229650efaf21d93107aa8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55548 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:05:12,416 - lava_callback - INFO - Received callback for node 69c2298b0efaf21d93107abf INFO: 172.21.0.1:39576 - "POST /node/69c2298b0efaf21d93107abf HTTP/1.0" 202 Accepted 2026-03-26 06:05:12,421 - lava_callback - INFO - Starting async job submit for node 69c2298b0efaf21d93107abf 2026-03-26 06:05:12,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:12,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2298b0efaf21d93107abf HTTP/1.1" 200 None 2026-03-26 06:05:12,438 - lava_callback - DEBUG - Processing node 69c2298b0efaf21d93107abf: baseline-arm-clabbe 2026-03-26 06:05:12,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2298b0efaf21d93107abf 2026-03-26 06:05:12,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:12,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2298b0efaf21d93107abf HTTP/1.1" 200 None 2026-03-26 06:05:12,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:12,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:05:12,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:12,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2298b0efaf21d93107abf HTTP/1.1" 200 None 2026-03-26 06:05:12,502 - lava_callback - ERROR - Error processing callback for node 69c2298b0efaf21d93107abf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33658 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:05:52,485 - lava_callback - INFO - Received callback for node 69c229b10efaf21d93107ac9 INFO: 172.21.0.1:40758 - "POST /node/69c229b10efaf21d93107ac9 HTTP/1.0" 202 Accepted 2026-03-26 06:05:52,490 - lava_callback - INFO - Starting async job submit for node 69c229b10efaf21d93107ac9 2026-03-26 06:05:52,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:52,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229b10efaf21d93107ac9 HTTP/1.1" 200 None 2026-03-26 06:05:52,507 - lava_callback - DEBUG - Processing node 69c229b10efaf21d93107ac9: baseline-arm-clabbe 2026-03-26 06:05:52,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c229b10efaf21d93107ac9 2026-03-26 06:05:52,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:52,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229b10efaf21d93107ac9 HTTP/1.1" 200 None 2026-03-26 06:05:52,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:52,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:05:52,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:05:52,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c229b10efaf21d93107ac9 HTTP/1.1" 200 None 2026-03-26 06:05:52,582 - lava_callback - ERROR - Error processing callback for node 69c229b10efaf21d93107ac9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57866 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:06:32,322 - lava_callback - INFO - Received callback for node 69c229d70efaf21d93107ada INFO: 172.21.0.1:56964 - "POST /node/69c229d70efaf21d93107ada HTTP/1.0" 202 Accepted 2026-03-26 06:06:32,328 - lava_callback - INFO - Starting async job submit for node 69c229d70efaf21d93107ada 2026-03-26 06:06:32,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:06:32,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229d70efaf21d93107ada HTTP/1.1" 200 None 2026-03-26 06:06:32,345 - lava_callback - DEBUG - Processing node 69c229d70efaf21d93107ada: baseline-arm-clabbe 2026-03-26 06:06:32,345 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c229d70efaf21d93107ada 2026-03-26 06:06:32,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:06:32,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229d70efaf21d93107ada HTTP/1.1" 200 None 2026-03-26 06:06:32,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:06:32,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:06:32,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:06:32,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c229d70efaf21d93107ada HTTP/1.1" 200 None 2026-03-26 06:06:32,420 - lava_callback - ERROR - Error processing callback for node 69c229d70efaf21d93107ada: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44250 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:07:12,444 - lava_callback - INFO - Received callback for node 69c229fe0efaf21d93107aec INFO: 172.21.0.1:49316 - "POST /node/69c229fe0efaf21d93107aec HTTP/1.0" 202 Accepted 2026-03-26 06:07:12,450 - lava_callback - INFO - Starting async job submit for node 69c229fe0efaf21d93107aec 2026-03-26 06:07:12,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:12,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229fe0efaf21d93107aec HTTP/1.1" 200 None 2026-03-26 06:07:12,466 - lava_callback - DEBUG - Processing node 69c229fe0efaf21d93107aec: baseline-arm-clabbe 2026-03-26 06:07:12,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c229fe0efaf21d93107aec 2026-03-26 06:07:12,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:12,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c229fe0efaf21d93107aec HTTP/1.1" 200 None 2026-03-26 06:07:12,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:12,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:07:12,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:12,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c229fe0efaf21d93107aec HTTP/1.1" 200 None 2026-03-26 06:07:12,547 - lava_callback - ERROR - Error processing callback for node 69c229fe0efaf21d93107aec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52278 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:07:52,442 - lava_callback - INFO - Received callback for node 69c22a230efaf21d93107afd INFO: 172.21.0.1:33036 - "POST /node/69c22a230efaf21d93107afd HTTP/1.0" 202 Accepted 2026-03-26 06:07:52,447 - lava_callback - INFO - Starting async job submit for node 69c22a230efaf21d93107afd 2026-03-26 06:07:52,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:52,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22a230efaf21d93107afd HTTP/1.1" 200 None 2026-03-26 06:07:52,464 - lava_callback - DEBUG - Processing node 69c22a230efaf21d93107afd: baseline-arm-clabbe 2026-03-26 06:07:52,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22a230efaf21d93107afd 2026-03-26 06:07:52,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:52,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22a230efaf21d93107afd HTTP/1.1" 200 None 2026-03-26 06:07:52,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:52,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:07:52,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:07:52,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22a230efaf21d93107afd HTTP/1.1" 200 None 2026-03-26 06:07:52,528 - lava_callback - ERROR - Error processing callback for node 69c22a230efaf21d93107afd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45350 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:08:12,478 - lava_callback - INFO - Received callback for node 69c22a490efaf21d93107b0d INFO: 172.21.0.1:54768 - "POST /node/69c22a490efaf21d93107b0d HTTP/1.0" 202 Accepted 2026-03-26 06:08:12,484 - lava_callback - INFO - Starting async job submit for node 69c22a490efaf21d93107b0d 2026-03-26 06:08:12,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:12,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22a490efaf21d93107b0d HTTP/1.1" 200 None 2026-03-26 06:08:12,499 - lava_callback - DEBUG - Processing node 69c22a490efaf21d93107b0d: baseline-arm-clabbe 2026-03-26 06:08:12,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22a490efaf21d93107b0d 2026-03-26 06:08:12,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:12,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22a490efaf21d93107b0d HTTP/1.1" 200 None 2026-03-26 06:08:12,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:12,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:08:12,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:12,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22a490efaf21d93107b0d HTTP/1.1" 200 None 2026-03-26 06:08:12,550 - lava_callback - ERROR - Error processing callback for node 69c22a490efaf21d93107b0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58392 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:08:52,488 - lava_callback - INFO - Received callback for node 69c22a6f0efaf21d93107b1c INFO: 172.21.0.1:41482 - "POST /node/69c22a6f0efaf21d93107b1c HTTP/1.0" 202 Accepted 2026-03-26 06:08:52,493 - lava_callback - INFO - Starting async job submit for node 69c22a6f0efaf21d93107b1c 2026-03-26 06:08:52,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:52,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22a6f0efaf21d93107b1c HTTP/1.1" 200 None 2026-03-26 06:08:52,509 - lava_callback - DEBUG - Processing node 69c22a6f0efaf21d93107b1c: baseline-arm-clabbe 2026-03-26 06:08:52,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22a6f0efaf21d93107b1c 2026-03-26 06:08:52,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:52,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22a6f0efaf21d93107b1c HTTP/1.1" 200 None 2026-03-26 06:08:52,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:52,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-26 06:08:52,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:08:52,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22a6f0efaf21d93107b1c HTTP/1.1" 200 None 2026-03-26 06:08:52,594 - lava_callback - ERROR - Error processing callback for node 69c22a6f0efaf21d93107b1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38902 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:16:27,859 - lava_callback - INFO - Received callback for node 69c22c320efaf21d93107b67 INFO: 172.21.0.1:38918 - "POST /node/69c22c320efaf21d93107b67 HTTP/1.0" 202 Accepted 2026-03-26 06:16:27,865 - lava_callback - INFO - Starting async job submit for node 69c22c320efaf21d93107b67 2026-03-26 06:16:27,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:16:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22c320efaf21d93107b67 HTTP/1.1" 200 None 2026-03-26 06:16:27,881 - lava_callback - DEBUG - Processing node 69c22c320efaf21d93107b67: baseline-arm64-baylibre 2026-03-26 06:16:27,882 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22c320efaf21d93107b67 2026-03-26 06:16:27,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:16:27,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22c320efaf21d93107b67 HTTP/1.1" 200 None 2026-03-26 06:16:27,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:16:27,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 06:16:27,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:16:27,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22c320efaf21d93107b67 HTTP/1.1" 200 None 2026-03-26 06:16:27,961 - lava_callback - ERROR - Error processing callback for node 69c22c320efaf21d93107b67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:17:47,630 - lava_callback - INFO - Received callback for node 69c22c870efaf21d93107b71 INFO: 172.21.0.1:37268 - "POST /node/69c22c870efaf21d93107b71 HTTP/1.0" 202 Accepted 2026-03-26 06:17:47,636 - lava_callback - INFO - Starting async job submit for node 69c22c870efaf21d93107b71 2026-03-26 06:17:47,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:17:47,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22c870efaf21d93107b71 HTTP/1.1" 200 None 2026-03-26 06:17:47,652 - lava_callback - DEBUG - Processing node 69c22c870efaf21d93107b71: baseline-arm64-baylibre 2026-03-26 06:17:47,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22c870efaf21d93107b71 2026-03-26 06:17:47,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:17:47,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22c870efaf21d93107b71 HTTP/1.1" 200 None 2026-03-26 06:17:47,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:17:47,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 06:17:47,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:17:47,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22c870efaf21d93107b71 HTTP/1.1" 200 None 2026-03-26 06:17:47,705 - lava_callback - ERROR - Error processing callback for node 69c22c870efaf21d93107b71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33914 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:19:28,241 - lava_callback - INFO - Received callback for node 69c22cdc0efaf21d93107b8b INFO: 172.21.0.1:33926 - "POST /node/69c22cdc0efaf21d93107b8b HTTP/1.0" 202 Accepted 2026-03-26 06:19:28,246 - lava_callback - INFO - Starting async job submit for node 69c22cdc0efaf21d93107b8b 2026-03-26 06:19:28,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:19:28,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22cdc0efaf21d93107b8b HTTP/1.1" 200 None 2026-03-26 06:19:28,264 - lava_callback - DEBUG - Processing node 69c22cdc0efaf21d93107b8b: baseline-arm64-baylibre 2026-03-26 06:19:28,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22cdc0efaf21d93107b8b 2026-03-26 06:19:28,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:19:28,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22cdc0efaf21d93107b8b HTTP/1.1" 200 None 2026-03-26 06:19:28,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:19:28,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 06:19:28,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:19:28,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22cdc0efaf21d93107b8b HTTP/1.1" 200 None 2026-03-26 06:19:28,331 - lava_callback - ERROR - Error processing callback for node 69c22cdc0efaf21d93107b8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41172 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:20:47,818 - lava_callback - INFO - Received callback for node 69c22d330efaf21d93107b9d INFO: 172.21.0.1:41188 - "POST /node/69c22d330efaf21d93107b9d HTTP/1.0" 202 Accepted 2026-03-26 06:20:47,824 - lava_callback - INFO - Starting async job submit for node 69c22d330efaf21d93107b9d 2026-03-26 06:20:47,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:20:47,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22d330efaf21d93107b9d HTTP/1.1" 200 None 2026-03-26 06:20:47,839 - lava_callback - DEBUG - Processing node 69c22d330efaf21d93107b9d: baseline-arm64-baylibre 2026-03-26 06:20:47,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22d330efaf21d93107b9d 2026-03-26 06:20:47,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:20:47,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22d330efaf21d93107b9d HTTP/1.1" 200 None 2026-03-26 06:20:47,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:20:47,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 06:20:47,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:20:47,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22d330efaf21d93107b9d HTTP/1.1" 200 None 2026-03-26 06:20:47,915 - lava_callback - ERROR - Error processing callback for node 69c22d330efaf21d93107b9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55638 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:22:08,248 - lava_callback - INFO - Received callback for node 69c22d890efaf21d93107baf INFO: 172.21.0.1:55644 - "POST /node/69c22d890efaf21d93107baf HTTP/1.0" 202 Accepted 2026-03-26 06:22:08,253 - lava_callback - INFO - Starting async job submit for node 69c22d890efaf21d93107baf 2026-03-26 06:22:08,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:22:08,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22d890efaf21d93107baf HTTP/1.1" 200 None 2026-03-26 06:22:08,270 - lava_callback - DEBUG - Processing node 69c22d890efaf21d93107baf: baseline-arm64-baylibre 2026-03-26 06:22:08,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22d890efaf21d93107baf 2026-03-26 06:22:08,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:22:08,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22d890efaf21d93107baf HTTP/1.1" 200 None 2026-03-26 06:22:08,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:22:08,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 06:22:08,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:22:08,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22d890efaf21d93107baf HTTP/1.1" 200 None 2026-03-26 06:22:08,329 - lava_callback - ERROR - Error processing callback for node 69c22d890efaf21d93107baf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53300 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:23:47,986 - lava_callback - INFO - Received callback for node 69c22de00efaf21d93107bc0 INFO: 172.21.0.1:53314 - "POST /node/69c22de00efaf21d93107bc0 HTTP/1.0" 202 Accepted 2026-03-26 06:23:47,991 - lava_callback - INFO - Starting async job submit for node 69c22de00efaf21d93107bc0 2026-03-26 06:23:47,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:23:48,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22de00efaf21d93107bc0 HTTP/1.1" 200 None 2026-03-26 06:23:48,014 - lava_callback - DEBUG - Processing node 69c22de00efaf21d93107bc0: baseline-arm64-baylibre 2026-03-26 06:23:48,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c22de00efaf21d93107bc0 2026-03-26 06:23:48,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:23:48,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c22de00efaf21d93107bc0 HTTP/1.1" 200 None 2026-03-26 06:23:48,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:23:48,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-26 06:23:48,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:23:48,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c22de00efaf21d93107bc0 HTTP/1.1" 200 None 2026-03-26 06:23:48,088 - lava_callback - ERROR - Error processing callback for node 69c22de00efaf21d93107bc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35912 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:42:54,080 - lava_callback - INFO - Received callback for node 69c232600efaf21d93107c7a INFO: 172.21.0.1:58458 - "POST /node/69c232600efaf21d93107c7a HTTP/1.0" 202 Accepted 2026-03-26 06:42:54,086 - lava_callback - INFO - Starting async job submit for node 69c232600efaf21d93107c7a 2026-03-26 06:42:54,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:42:54,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232600efaf21d93107c7a HTTP/1.1" 200 None 2026-03-26 06:42:54,103 - lava_callback - DEBUG - Processing node 69c232600efaf21d93107c7a: baseline-arm-clabbe 2026-03-26 06:42:54,103 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c232600efaf21d93107c7a 2026-03-26 06:42:54,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:42:54,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232600efaf21d93107c7a HTTP/1.1" 200 None 2026-03-26 06:42:54,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:42:54,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 06:42:54,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:42:54,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c232600efaf21d93107c7a HTTP/1.1" 200 None 2026-03-26 06:42:54,160 - lava_callback - ERROR - Error processing callback for node 69c232600efaf21d93107c7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:43:33,990 - lava_callback - INFO - Received callback for node 69c232850efaf21d93107c89 INFO: 172.21.0.1:57052 - "POST /node/69c232850efaf21d93107c89 HTTP/1.0" 202 Accepted 2026-03-26 06:43:33,996 - lava_callback - INFO - Starting async job submit for node 69c232850efaf21d93107c89 2026-03-26 06:43:33,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:43:34,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232850efaf21d93107c89 HTTP/1.1" 200 None 2026-03-26 06:43:34,012 - lava_callback - DEBUG - Processing node 69c232850efaf21d93107c89: baseline-arm-clabbe 2026-03-26 06:43:34,012 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c232850efaf21d93107c89 2026-03-26 06:43:34,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:43:34,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232850efaf21d93107c89 HTTP/1.1" 200 None 2026-03-26 06:43:34,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:43:34,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 06:43:34,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:43:34,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c232850efaf21d93107c89 HTTP/1.1" 200 None 2026-03-26 06:43:34,092 - lava_callback - ERROR - Error processing callback for node 69c232850efaf21d93107c89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46542 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:44:14,080 - lava_callback - INFO - Received callback for node 69c232aa0efaf21d93107c98 INFO: 172.21.0.1:38388 - "POST /node/69c232aa0efaf21d93107c98 HTTP/1.0" 202 Accepted 2026-03-26 06:44:14,085 - lava_callback - INFO - Starting async job submit for node 69c232aa0efaf21d93107c98 2026-03-26 06:44:14,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:14,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232aa0efaf21d93107c98 HTTP/1.1" 200 None 2026-03-26 06:44:14,123 - lava_callback - DEBUG - Processing node 69c232aa0efaf21d93107c98: baseline-arm-clabbe 2026-03-26 06:44:14,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c232aa0efaf21d93107c98 2026-03-26 06:44:14,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:14,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232aa0efaf21d93107c98 HTTP/1.1" 200 None 2026-03-26 06:44:14,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:14,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 06:44:14,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:14,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c232aa0efaf21d93107c98 HTTP/1.1" 200 None 2026-03-26 06:44:14,203 - lava_callback - ERROR - Error processing callback for node 69c232aa0efaf21d93107c98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:44:34,046 - lava_callback - INFO - Received callback for node 69c232cf0efaf21d93107caa INFO: 172.21.0.1:54222 - "POST /node/69c232cf0efaf21d93107caa HTTP/1.0" 202 Accepted 2026-03-26 06:44:34,052 - lava_callback - INFO - Starting async job submit for node 69c232cf0efaf21d93107caa 2026-03-26 06:44:34,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:34,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232cf0efaf21d93107caa HTTP/1.1" 200 None 2026-03-26 06:44:34,069 - lava_callback - DEBUG - Processing node 69c232cf0efaf21d93107caa: baseline-arm-clabbe 2026-03-26 06:44:34,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c232cf0efaf21d93107caa 2026-03-26 06:44:34,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:34,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232cf0efaf21d93107caa HTTP/1.1" 200 None 2026-03-26 06:44:34,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:34,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 06:44:34,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:44:34,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c232cf0efaf21d93107caa HTTP/1.1" 200 None 2026-03-26 06:44:34,126 - lava_callback - ERROR - Error processing callback for node 69c232cf0efaf21d93107caa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54402 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:45:14,086 - lava_callback - INFO - Received callback for node 69c232f40efaf21d93107cb6 INFO: 172.21.0.1:41568 - "POST /node/69c232f40efaf21d93107cb6 HTTP/1.0" 202 Accepted 2026-03-26 06:45:14,092 - lava_callback - INFO - Starting async job submit for node 69c232f40efaf21d93107cb6 2026-03-26 06:45:14,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:45:14,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232f40efaf21d93107cb6 HTTP/1.1" 200 None 2026-03-26 06:45:14,108 - lava_callback - DEBUG - Processing node 69c232f40efaf21d93107cb6: baseline-arm-clabbe 2026-03-26 06:45:14,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c232f40efaf21d93107cb6 2026-03-26 06:45:14,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:45:14,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c232f40efaf21d93107cb6 HTTP/1.1" 200 None 2026-03-26 06:45:14,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:45:14,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-26 06:45:14,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:45:14,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c232f40efaf21d93107cb6 HTTP/1.1" 200 None 2026-03-26 06:45:14,167 - lava_callback - ERROR - Error processing callback for node 69c232f40efaf21d93107cb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57728 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:46:14,189 - lava_callback - INFO - Received callback for node 69c233260efaf21d93107cce INFO: 172.21.0.1:43116 - "POST /node/69c233260efaf21d93107cce HTTP/1.0" 202 Accepted 2026-03-26 06:46:14,195 - lava_callback - INFO - Starting async job submit for node 69c233260efaf21d93107cce 2026-03-26 06:46:14,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:14,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233260efaf21d93107cce HTTP/1.1" 200 None 2026-03-26 06:46:14,229 - lava_callback - DEBUG - Processing node 69c233260efaf21d93107cce: baseline-arm-clabbe 2026-03-26 06:46:14,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c233260efaf21d93107cce 2026-03-26 06:46:14,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:14,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233260efaf21d93107cce HTTP/1.1" 200 None 2026-03-26 06:46:14,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:14,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:46:14,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:14,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c233260efaf21d93107cce HTTP/1.1" 200 None 2026-03-26 06:46:14,281 - lava_callback - ERROR - Error processing callback for node 69c233260efaf21d93107cce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32890 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:46:54,163 - lava_callback - INFO - Received callback for node 69c2334e0efaf21d93107cd8 INFO: 172.21.0.1:41542 - "POST /node/69c2334e0efaf21d93107cd8 HTTP/1.0" 202 Accepted 2026-03-26 06:46:54,169 - lava_callback - INFO - Starting async job submit for node 69c2334e0efaf21d93107cd8 2026-03-26 06:46:54,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:54,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2334e0efaf21d93107cd8 HTTP/1.1" 200 None 2026-03-26 06:46:54,201 - lava_callback - DEBUG - Processing node 69c2334e0efaf21d93107cd8: baseline-arm-clabbe 2026-03-26 06:46:54,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2334e0efaf21d93107cd8 2026-03-26 06:46:54,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:54,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2334e0efaf21d93107cd8 HTTP/1.1" 200 None 2026-03-26 06:46:54,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:54,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:46:54,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:46:54,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2334e0efaf21d93107cd8 HTTP/1.1" 200 None 2026-03-26 06:46:54,262 - lava_callback - ERROR - Error processing callback for node 69c2334e0efaf21d93107cd8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41540 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:47:34,211 - lava_callback - INFO - Received callback for node 69c233760efaf21d93107ce9 INFO: 172.21.0.1:41994 - "POST /node/69c233760efaf21d93107ce9 HTTP/1.0" 202 Accepted 2026-03-26 06:47:34,216 - lava_callback - INFO - Starting async job submit for node 69c233760efaf21d93107ce9 2026-03-26 06:47:34,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:47:34,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233760efaf21d93107ce9 HTTP/1.1" 200 None 2026-03-26 06:47:34,232 - lava_callback - DEBUG - Processing node 69c233760efaf21d93107ce9: baseline-arm-clabbe 2026-03-26 06:47:34,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c233760efaf21d93107ce9 2026-03-26 06:47:34,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:47:34,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233760efaf21d93107ce9 HTTP/1.1" 200 None 2026-03-26 06:47:34,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:47:34,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:47:34,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:47:34,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c233760efaf21d93107ce9 HTTP/1.1" 200 None 2026-03-26 06:47:34,289 - lava_callback - ERROR - Error processing callback for node 69c233760efaf21d93107ce9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:48:14,222 - lava_callback - INFO - Received callback for node 69c2339e0efaf21d93107d02 INFO: 172.21.0.1:51014 - "POST /node/69c2339e0efaf21d93107d02 HTTP/1.0" 202 Accepted 2026-03-26 06:48:14,228 - lava_callback - INFO - Starting async job submit for node 69c2339e0efaf21d93107d02 2026-03-26 06:48:14,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:14,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2339e0efaf21d93107d02 HTTP/1.1" 200 None 2026-03-26 06:48:14,246 - lava_callback - DEBUG - Processing node 69c2339e0efaf21d93107d02: baseline-arm-clabbe 2026-03-26 06:48:14,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2339e0efaf21d93107d02 2026-03-26 06:48:14,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:14,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2339e0efaf21d93107d02 HTTP/1.1" 200 None 2026-03-26 06:48:14,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:14,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:48:14,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:14,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2339e0efaf21d93107d02 HTTP/1.1" 200 None 2026-03-26 06:48:14,313 - lava_callback - ERROR - Error processing callback for node 69c2339e0efaf21d93107d02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:48:54,319 - lava_callback - INFO - Received callback for node 69c233c30efaf21d93107d0b INFO: 172.21.0.1:37146 - "POST /node/69c233c30efaf21d93107d0b HTTP/1.0" 202 Accepted 2026-03-26 06:48:54,324 - lava_callback - INFO - Starting async job submit for node 69c233c30efaf21d93107d0b 2026-03-26 06:48:54,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:54,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233c30efaf21d93107d0b HTTP/1.1" 200 None 2026-03-26 06:48:54,341 - lava_callback - DEBUG - Processing node 69c233c30efaf21d93107d0b: baseline-arm-clabbe 2026-03-26 06:48:54,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c233c30efaf21d93107d0b 2026-03-26 06:48:54,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:54,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233c30efaf21d93107d0b HTTP/1.1" 200 None 2026-03-26 06:48:54,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:54,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:48:54,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:48:54,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c233c30efaf21d93107d0b HTTP/1.1" 200 None 2026-03-26 06:48:54,392 - lava_callback - ERROR - Error processing callback for node 69c233c30efaf21d93107d0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42142 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:49:34,263 - lava_callback - INFO - Received callback for node 69c233e80efaf21d93107d1f INFO: 172.21.0.1:56232 - "POST /node/69c233e80efaf21d93107d1f HTTP/1.0" 202 Accepted 2026-03-26 06:49:34,268 - lava_callback - INFO - Starting async job submit for node 69c233e80efaf21d93107d1f 2026-03-26 06:49:34,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:34,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233e80efaf21d93107d1f HTTP/1.1" 200 None 2026-03-26 06:49:34,285 - lava_callback - DEBUG - Processing node 69c233e80efaf21d93107d1f: baseline-arm-clabbe 2026-03-26 06:49:34,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c233e80efaf21d93107d1f 2026-03-26 06:49:34,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:34,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c233e80efaf21d93107d1f HTTP/1.1" 200 None 2026-03-26 06:49:34,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:34,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:49:34,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:34,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c233e80efaf21d93107d1f HTTP/1.1" 200 None 2026-03-26 06:49:34,388 - lava_callback - ERROR - Error processing callback for node 69c233e80efaf21d93107d1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53090 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:49:54,340 - lava_callback - INFO - Received callback for node 69c2340c0efaf21d93107d33 INFO: 172.21.0.1:51746 - "POST /node/69c2340c0efaf21d93107d33 HTTP/1.0" 202 Accepted 2026-03-26 06:49:54,346 - lava_callback - INFO - Starting async job submit for node 69c2340c0efaf21d93107d33 2026-03-26 06:49:54,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:54,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2340c0efaf21d93107d33 HTTP/1.1" 200 None 2026-03-26 06:49:54,367 - lava_callback - DEBUG - Processing node 69c2340c0efaf21d93107d33: baseline-arm-clabbe 2026-03-26 06:49:54,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2340c0efaf21d93107d33 2026-03-26 06:49:54,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:54,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2340c0efaf21d93107d33 HTTP/1.1" 200 None 2026-03-26 06:49:54,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:54,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:49:54,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:49:54,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2340c0efaf21d93107d33 HTTP/1.1" 200 None 2026-03-26 06:49:54,425 - lava_callback - ERROR - Error processing callback for node 69c2340c0efaf21d93107d33: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33854 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:50:34,387 - lava_callback - INFO - Received callback for node 69c234310efaf21d93107d44 INFO: 172.21.0.1:41724 - "POST /node/69c234310efaf21d93107d44 HTTP/1.0" 202 Accepted 2026-03-26 06:50:34,393 - lava_callback - INFO - Starting async job submit for node 69c234310efaf21d93107d44 2026-03-26 06:50:34,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:50:34,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234310efaf21d93107d44 HTTP/1.1" 200 None 2026-03-26 06:50:34,438 - lava_callback - DEBUG - Processing node 69c234310efaf21d93107d44: baseline-arm-clabbe 2026-03-26 06:50:34,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c234310efaf21d93107d44 2026-03-26 06:50:34,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:50:34,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234310efaf21d93107d44 HTTP/1.1" 200 None 2026-03-26 06:50:34,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:50:34,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:50:34,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:50:34,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c234310efaf21d93107d44 HTTP/1.1" 200 None 2026-03-26 06:50:34,515 - lava_callback - ERROR - Error processing callback for node 69c234310efaf21d93107d44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59512 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:51:14,376 - lava_callback - INFO - Received callback for node 69c234560efaf21d93107d5d INFO: 172.21.0.1:35940 - "POST /node/69c234560efaf21d93107d5d HTTP/1.0" 202 Accepted 2026-03-26 06:51:14,382 - lava_callback - INFO - Starting async job submit for node 69c234560efaf21d93107d5d 2026-03-26 06:51:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:14,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234560efaf21d93107d5d HTTP/1.1" 200 None 2026-03-26 06:51:14,397 - lava_callback - DEBUG - Processing node 69c234560efaf21d93107d5d: baseline-arm-clabbe 2026-03-26 06:51:14,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c234560efaf21d93107d5d 2026-03-26 06:51:14,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:14,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234560efaf21d93107d5d HTTP/1.1" 200 None 2026-03-26 06:51:14,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:14,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:51:14,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:14,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c234560efaf21d93107d5d HTTP/1.1" 200 None 2026-03-26 06:51:14,468 - lava_callback - ERROR - Error processing callback for node 69c234560efaf21d93107d5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39882 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:51:54,362 - lava_callback - INFO - Received callback for node 69c2347a0efaf21d93107d6c INFO: 172.21.0.1:37186 - "POST /node/69c2347a0efaf21d93107d6c HTTP/1.0" 202 Accepted 2026-03-26 06:51:54,368 - lava_callback - INFO - Starting async job submit for node 69c2347a0efaf21d93107d6c 2026-03-26 06:51:54,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:54,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2347a0efaf21d93107d6c HTTP/1.1" 200 None 2026-03-26 06:51:54,384 - lava_callback - DEBUG - Processing node 69c2347a0efaf21d93107d6c: baseline-arm-clabbe 2026-03-26 06:51:54,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2347a0efaf21d93107d6c 2026-03-26 06:51:54,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:54,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2347a0efaf21d93107d6c HTTP/1.1" 200 None 2026-03-26 06:51:54,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:54,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:51:54,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:51:54,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2347a0efaf21d93107d6c HTTP/1.1" 200 None 2026-03-26 06:51:54,447 - lava_callback - ERROR - Error processing callback for node 69c2347a0efaf21d93107d6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:52:34,456 - lava_callback - INFO - Received callback for node 69c2349f0efaf21d93107d80 INFO: 172.21.0.1:56858 - "POST /node/69c2349f0efaf21d93107d80 HTTP/1.0" 202 Accepted 2026-03-26 06:52:34,461 - lava_callback - INFO - Starting async job submit for node 69c2349f0efaf21d93107d80 2026-03-26 06:52:34,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:52:34,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2349f0efaf21d93107d80 HTTP/1.1" 200 None 2026-03-26 06:52:34,479 - lava_callback - DEBUG - Processing node 69c2349f0efaf21d93107d80: baseline-arm-clabbe 2026-03-26 06:52:34,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2349f0efaf21d93107d80 2026-03-26 06:52:34,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:52:34,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2349f0efaf21d93107d80 HTTP/1.1" 200 None 2026-03-26 06:52:34,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:52:34,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:52:34,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:52:34,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2349f0efaf21d93107d80 HTTP/1.1" 200 None 2026-03-26 06:52:34,540 - lava_callback - ERROR - Error processing callback for node 69c2349f0efaf21d93107d80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56164 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:53:14,409 - lava_callback - INFO - Received callback for node 69c234c40efaf21d93107d91 INFO: 172.21.0.1:40904 - "POST /node/69c234c40efaf21d93107d91 HTTP/1.0" 202 Accepted 2026-03-26 06:53:14,415 - lava_callback - INFO - Starting async job submit for node 69c234c40efaf21d93107d91 2026-03-26 06:53:14,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:14,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234c40efaf21d93107d91 HTTP/1.1" 200 None 2026-03-26 06:53:14,432 - lava_callback - DEBUG - Processing node 69c234c40efaf21d93107d91: baseline-arm-clabbe 2026-03-26 06:53:14,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c234c40efaf21d93107d91 2026-03-26 06:53:14,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:14,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234c40efaf21d93107d91 HTTP/1.1" 200 None 2026-03-26 06:53:14,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:14,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:53:14,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:14,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c234c40efaf21d93107d91 HTTP/1.1" 200 None 2026-03-26 06:53:14,498 - lava_callback - ERROR - Error processing callback for node 69c234c40efaf21d93107d91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41768 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:53:34,497 - lava_callback - INFO - Received callback for node 69c234e90efaf21d93107da9 INFO: 172.21.0.1:36818 - "POST /node/69c234e90efaf21d93107da9 HTTP/1.0" 202 Accepted 2026-03-26 06:53:34,503 - lava_callback - INFO - Starting async job submit for node 69c234e90efaf21d93107da9 2026-03-26 06:53:34,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:34,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234e90efaf21d93107da9 HTTP/1.1" 200 None 2026-03-26 06:53:34,519 - lava_callback - DEBUG - Processing node 69c234e90efaf21d93107da9: baseline-arm-clabbe 2026-03-26 06:53:34,519 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c234e90efaf21d93107da9 2026-03-26 06:53:34,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:34,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c234e90efaf21d93107da9 HTTP/1.1" 200 None 2026-03-26 06:53:34,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:34,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:53:34,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:53:34,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c234e90efaf21d93107da9 HTTP/1.1" 200 None 2026-03-26 06:53:34,579 - lava_callback - ERROR - Error processing callback for node 69c234e90efaf21d93107da9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:54:14,496 - lava_callback - INFO - Received callback for node 69c2350d0efaf21d93107db3 INFO: 172.21.0.1:34262 - "POST /node/69c2350d0efaf21d93107db3 HTTP/1.0" 202 Accepted 2026-03-26 06:54:14,502 - lava_callback - INFO - Starting async job submit for node 69c2350d0efaf21d93107db3 2026-03-26 06:54:14,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:14,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2350d0efaf21d93107db3 HTTP/1.1" 200 None 2026-03-26 06:54:14,518 - lava_callback - DEBUG - Processing node 69c2350d0efaf21d93107db3: baseline-arm-clabbe 2026-03-26 06:54:14,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2350d0efaf21d93107db3 2026-03-26 06:54:14,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:14,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2350d0efaf21d93107db3 HTTP/1.1" 200 None 2026-03-26 06:54:14,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:14,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:54:14,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:14,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2350d0efaf21d93107db3 HTTP/1.1" 200 None 2026-03-26 06:54:14,575 - lava_callback - ERROR - Error processing callback for node 69c2350d0efaf21d93107db3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:54:54,560 - lava_callback - INFO - Received callback for node 69c235320efaf21d93107dc4 INFO: 172.21.0.1:43816 - "POST /node/69c235320efaf21d93107dc4 HTTP/1.0" 202 Accepted 2026-03-26 06:54:54,566 - lava_callback - INFO - Starting async job submit for node 69c235320efaf21d93107dc4 2026-03-26 06:54:54,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:54,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235320efaf21d93107dc4 HTTP/1.1" 200 None 2026-03-26 06:54:54,584 - lava_callback - DEBUG - Processing node 69c235320efaf21d93107dc4: baseline-arm-clabbe 2026-03-26 06:54:54,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c235320efaf21d93107dc4 2026-03-26 06:54:54,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:54,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235320efaf21d93107dc4 HTTP/1.1" 200 None 2026-03-26 06:54:54,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:54,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-26 06:54:54,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:54:54,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c235320efaf21d93107dc4 HTTP/1.1" 200 None 2026-03-26 06:54:54,677 - lava_callback - ERROR - Error processing callback for node 69c235320efaf21d93107dc4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60466 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:55:34,548 - lava_callback - INFO - Received callback for node 69c235570efaf21d93107dd6 INFO: 172.21.0.1:35130 - "POST /node/69c235570efaf21d93107dd6 HTTP/1.0" 202 Accepted 2026-03-26 06:55:34,554 - lava_callback - INFO - Starting async job submit for node 69c235570efaf21d93107dd6 2026-03-26 06:55:34,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:55:34,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235570efaf21d93107dd6 HTTP/1.1" 200 None 2026-03-26 06:55:34,571 - lava_callback - DEBUG - Processing node 69c235570efaf21d93107dd6: baseline-arm-clabbe 2026-03-26 06:55:34,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c235570efaf21d93107dd6 2026-03-26 06:55:34,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:55:34,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235570efaf21d93107dd6 HTTP/1.1" 200 None 2026-03-26 06:55:34,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:55:34,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:55:34,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:55:34,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c235570efaf21d93107dd6 HTTP/1.1" 200 None 2026-03-26 06:55:34,625 - lava_callback - ERROR - Error processing callback for node 69c235570efaf21d93107dd6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35782 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:56:14,597 - lava_callback - INFO - Received callback for node 69c2357b0efaf21d93107de8 INFO: 172.21.0.1:40824 - "POST /node/69c2357b0efaf21d93107de8 HTTP/1.0" 202 Accepted 2026-03-26 06:56:14,602 - lava_callback - INFO - Starting async job submit for node 69c2357b0efaf21d93107de8 2026-03-26 06:56:14,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:14,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2357b0efaf21d93107de8 HTTP/1.1" 200 None 2026-03-26 06:56:14,619 - lava_callback - DEBUG - Processing node 69c2357b0efaf21d93107de8: baseline-arm-clabbe 2026-03-26 06:56:14,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2357b0efaf21d93107de8 2026-03-26 06:56:14,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:14,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2357b0efaf21d93107de8 HTTP/1.1" 200 None 2026-03-26 06:56:14,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:14,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:56:14,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:14,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2357b0efaf21d93107de8 HTTP/1.1" 200 None 2026-03-26 06:56:14,689 - lava_callback - ERROR - Error processing callback for node 69c2357b0efaf21d93107de8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60564 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:56:34,559 - lava_callback - INFO - Received callback for node 69c235a00efaf21d93107dfa INFO: 172.21.0.1:42522 - "POST /node/69c235a00efaf21d93107dfa HTTP/1.0" 202 Accepted 2026-03-26 06:56:34,564 - lava_callback - INFO - Starting async job submit for node 69c235a00efaf21d93107dfa 2026-03-26 06:56:34,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:34,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235a00efaf21d93107dfa HTTP/1.1" 200 None 2026-03-26 06:56:34,581 - lava_callback - DEBUG - Processing node 69c235a00efaf21d93107dfa: baseline-arm-clabbe 2026-03-26 06:56:34,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c235a00efaf21d93107dfa 2026-03-26 06:56:34,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:34,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235a00efaf21d93107dfa HTTP/1.1" 200 None 2026-03-26 06:56:34,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:34,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:56:34,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:56:34,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c235a00efaf21d93107dfa HTTP/1.1" 200 None 2026-03-26 06:56:34,643 - lava_callback - ERROR - Error processing callback for node 69c235a00efaf21d93107dfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38192 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:57:34,609 - lava_callback - INFO - Received callback for node 69c235ca0efaf21d93107e10 INFO: 172.21.0.1:43058 - "POST /node/69c235ca0efaf21d93107e10 HTTP/1.0" 202 Accepted 2026-03-26 06:57:34,614 - lava_callback - INFO - Starting async job submit for node 69c235ca0efaf21d93107e10 2026-03-26 06:57:34,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:57:34,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235ca0efaf21d93107e10 HTTP/1.1" 200 None 2026-03-26 06:57:34,631 - lava_callback - DEBUG - Processing node 69c235ca0efaf21d93107e10: baseline-arm-clabbe 2026-03-26 06:57:34,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c235ca0efaf21d93107e10 2026-03-26 06:57:34,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:57:34,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235ca0efaf21d93107e10 HTTP/1.1" 200 None 2026-03-26 06:57:34,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:57:34,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:57:34,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:57:34,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c235ca0efaf21d93107e10 HTTP/1.1" 200 None 2026-03-26 06:57:34,687 - lava_callback - ERROR - Error processing callback for node 69c235ca0efaf21d93107e10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50560 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:58:14,604 - lava_callback - INFO - Received callback for node 69c235f40efaf21d93107e24 INFO: 172.21.0.1:56718 - "POST /node/69c235f40efaf21d93107e24 HTTP/1.0" 202 Accepted 2026-03-26 06:58:14,609 - lava_callback - INFO - Starting async job submit for node 69c235f40efaf21d93107e24 2026-03-26 06:58:14,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:14,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235f40efaf21d93107e24 HTTP/1.1" 200 None 2026-03-26 06:58:14,626 - lava_callback - DEBUG - Processing node 69c235f40efaf21d93107e24: baseline-arm-clabbe 2026-03-26 06:58:14,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c235f40efaf21d93107e24 2026-03-26 06:58:14,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:14,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c235f40efaf21d93107e24 HTTP/1.1" 200 None 2026-03-26 06:58:14,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:14,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:58:14,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:14,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c235f40efaf21d93107e24 HTTP/1.1" 200 None 2026-03-26 06:58:14,684 - lava_callback - ERROR - Error processing callback for node 69c235f40efaf21d93107e24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:58:54,686 - lava_callback - INFO - Received callback for node 69c236190efaf21d93107e33 INFO: 172.21.0.1:47180 - "POST /node/69c236190efaf21d93107e33 HTTP/1.0" 202 Accepted 2026-03-26 06:58:54,692 - lava_callback - INFO - Starting async job submit for node 69c236190efaf21d93107e33 2026-03-26 06:58:54,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:54,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c236190efaf21d93107e33 HTTP/1.1" 200 None 2026-03-26 06:58:54,732 - lava_callback - DEBUG - Processing node 69c236190efaf21d93107e33: baseline-arm-clabbe 2026-03-26 06:58:54,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c236190efaf21d93107e33 2026-03-26 06:58:54,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:54,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c236190efaf21d93107e33 HTTP/1.1" 200 None 2026-03-26 06:58:54,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:54,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:58:54,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:58:54,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c236190efaf21d93107e33 HTTP/1.1" 200 None 2026-03-26 06:58:54,790 - lava_callback - ERROR - Error processing callback for node 69c236190efaf21d93107e33: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59106 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:59:14,649 - lava_callback - INFO - Received callback for node 69c2363e0efaf21d93107e3d INFO: 172.21.0.1:53784 - "POST /node/69c2363e0efaf21d93107e3d HTTP/1.0" 202 Accepted 2026-03-26 06:59:14,655 - lava_callback - INFO - Starting async job submit for node 69c2363e0efaf21d93107e3d 2026-03-26 06:59:14,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:14,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2363e0efaf21d93107e3d HTTP/1.1" 200 None 2026-03-26 06:59:14,679 - lava_callback - DEBUG - Processing node 69c2363e0efaf21d93107e3d: baseline-arm-clabbe 2026-03-26 06:59:14,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2363e0efaf21d93107e3d 2026-03-26 06:59:14,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:14,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2363e0efaf21d93107e3d HTTP/1.1" 200 None 2026-03-26 06:59:14,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:14,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:59:14,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:14,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2363e0efaf21d93107e3d HTTP/1.1" 200 None 2026-03-26 06:59:14,739 - lava_callback - ERROR - Error processing callback for node 69c2363e0efaf21d93107e3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42384 - "GET / HTTP/1.0" 200 OK 2026-03-26 06:59:54,721 - lava_callback - INFO - Received callback for node 69c236640efaf21d93107e49 INFO: 172.21.0.1:35344 - "POST /node/69c236640efaf21d93107e49 HTTP/1.0" 202 Accepted 2026-03-26 06:59:54,727 - lava_callback - INFO - Starting async job submit for node 69c236640efaf21d93107e49 2026-03-26 06:59:54,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:54,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c236640efaf21d93107e49 HTTP/1.1" 200 None 2026-03-26 06:59:54,746 - lava_callback - DEBUG - Processing node 69c236640efaf21d93107e49: baseline-arm-clabbe 2026-03-26 06:59:54,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c236640efaf21d93107e49 2026-03-26 06:59:54,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:54,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c236640efaf21d93107e49 HTTP/1.1" 200 None 2026-03-26 06:59:54,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:54,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 06:59:54,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 06:59:54,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c236640efaf21d93107e49 HTTP/1.1" 200 None 2026-03-26 06:59:54,800 - lava_callback - ERROR - Error processing callback for node 69c236640efaf21d93107e49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58444 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:00:24,230 - lava_callback - INFO - Received callback for node 69c411a8944faaf0e7f60e44 INFO: 172.21.0.1:50572 - "POST /node/69c411a8944faaf0e7f60e44 HTTP/1.0" 202 Accepted 2026-03-26 07:00:24,236 - lava_callback - INFO - Starting async job submit for node 69c411a8944faaf0e7f60e44 2026-03-26 07:00:24,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:24,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c411a8944faaf0e7f60e44 HTTP/1.1" 200 None 2026-03-26 07:00:24,282 - lava_callback - DEBUG - Processing node 69c411a8944faaf0e7f60e44: baseline-arm64-broonie 2026-03-26 07:00:24,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 07:00:24,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 07:00:24,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a8944faaf0e7f60e44/log.txt.gz 2026-03-26 07:00:25,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 07:00:25,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 07:00:25,282 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a8944faaf0e7f60e44/lava-logs.yaml 2026-03-26 07:00:25,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 07:00:25,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 07:00:25,450 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c411a8944faaf0e7f60e44/lava_callback.json.gz 2026-03-26 07:00:25,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:25,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c411a8944faaf0e7f60e44 HTTP/1.1" 200 None 2026-03-26 07:00:25,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:25,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-26 07:00:25,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:25,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c411a8944faaf0e7f60e44 HTTP/1.1" 200 None 2026-03-26 07:00:25,534 - lava_callback - INFO - Completed processing callback for node 69c411a8944faaf0e7f60e44 INFO: 172.21.0.1:50586 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:00:30,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:30,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-26 07:00:34,708 - lava_callback - INFO - Received callback for node 69c2368e0efaf21d93107e5e INFO: 172.21.0.1:36418 - "POST /node/69c2368e0efaf21d93107e5e HTTP/1.0" 202 Accepted 2026-03-26 07:00:34,715 - lava_callback - INFO - Starting async job submit for node 69c2368e0efaf21d93107e5e 2026-03-26 07:00:34,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:34,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2368e0efaf21d93107e5e HTTP/1.1" 200 None 2026-03-26 07:00:34,732 - lava_callback - DEBUG - Processing node 69c2368e0efaf21d93107e5e: baseline-arm-clabbe 2026-03-26 07:00:34,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2368e0efaf21d93107e5e 2026-03-26 07:00:34,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:34,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2368e0efaf21d93107e5e HTTP/1.1" 200 None 2026-03-26 07:00:34,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:34,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 07:00:34,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:00:34,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2368e0efaf21d93107e5e HTTP/1.1" 200 None 2026-03-26 07:00:34,800 - lava_callback - ERROR - Error processing callback for node 69c2368e0efaf21d93107e5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50838 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:01:14,794 - lava_callback - INFO - Received callback for node 69c236b50efaf21d93107e68 INFO: 172.21.0.1:59734 - "POST /node/69c236b50efaf21d93107e68 HTTP/1.0" 202 Accepted 2026-03-26 07:01:14,799 - lava_callback - INFO - Starting async job submit for node 69c236b50efaf21d93107e68 2026-03-26 07:01:14,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:01:14,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c236b50efaf21d93107e68 HTTP/1.1" 200 None 2026-03-26 07:01:14,816 - lava_callback - DEBUG - Processing node 69c236b50efaf21d93107e68: baseline-arm-clabbe 2026-03-26 07:01:14,816 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c236b50efaf21d93107e68 2026-03-26 07:01:14,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:01:14,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c236b50efaf21d93107e68 HTTP/1.1" 200 None 2026-03-26 07:01:14,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:01:14,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-26 07:01:14,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:01:14,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c236b50efaf21d93107e68 HTTP/1.1" 200 None 2026-03-26 07:01:14,877 - lava_callback - ERROR - Error processing callback for node 69c236b50efaf21d93107e68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49316 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:02:24,073 - lava_callback - INFO - Received callback for node 69c48873590270ad7b6d9587 INFO: 172.21.0.1:58038 - "POST /node/69c48873590270ad7b6d9587 HTTP/1.0" 202 Accepted 2026-03-26 07:02:24,080 - lava_callback - INFO - Starting async job submit for node 69c48873590270ad7b6d9587 2026-03-26 07:02:24,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:02:24,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48873590270ad7b6d9587 HTTP/1.1" 200 None 2026-03-26 07:02:24,117 - lava_callback - DEBUG - Processing node 69c48873590270ad7b6d9587: baseline-arm64-broonie 2026-03-26 07:02:24,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 07:02:24,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 07:02:24,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48873590270ad7b6d9587/log.txt.gz 2026-03-26 07:02:24,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 07:02:24,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 07:02:24,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48873590270ad7b6d9587/lava-logs.yaml 2026-03-26 07:02:24,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-26 07:02:25,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-26 07:02:25,095 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c48873590270ad7b6d9587/lava_callback.json.gz 2026-03-26 07:02:25,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:02:25,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48873590270ad7b6d9587 HTTP/1.1" 200 None 2026-03-26 07:02:25,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:02:25,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-26 07:02:25,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:02:25,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48873590270ad7b6d9587 HTTP/1.1" 200 None 2026-03-26 07:02:25,174 - lava_callback - INFO - Completed processing callback for node 69c48873590270ad7b6d9587 INFO: 172.21.0.1:58042 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:02:30,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:02:30,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58542 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:14:30,965 - lava_callback - INFO - Received callback for node 69c239ca0efaf21d93107f00 INFO: 172.21.0.1:43664 - "POST /node/69c239ca0efaf21d93107f00 HTTP/1.0" 202 Accepted 2026-03-26 07:14:30,972 - lava_callback - INFO - Starting async job submit for node 69c239ca0efaf21d93107f00 2026-03-26 07:14:30,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:14:30,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c239ca0efaf21d93107f00 HTTP/1.1" 200 None 2026-03-26 07:14:30,995 - lava_callback - DEBUG - Processing node 69c239ca0efaf21d93107f00: baseline-arm64-baylibre 2026-03-26 07:14:30,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c239ca0efaf21d93107f00 2026-03-26 07:14:30,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:14:31,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c239ca0efaf21d93107f00 HTTP/1.1" 200 None 2026-03-26 07:14:31,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:14:31,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:14:31,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:14:31,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c239ca0efaf21d93107f00 HTTP/1.1" 200 None 2026-03-26 07:14:31,065 - lava_callback - ERROR - Error processing callback for node 69c239ca0efaf21d93107f00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33256 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:15:51,059 - lava_callback - INFO - Received callback for node 69c23a1f0efaf21d93107f08 INFO: 172.21.0.1:48712 - "POST /node/69c23a1f0efaf21d93107f08 HTTP/1.0" 202 Accepted 2026-03-26 07:15:51,064 - lava_callback - INFO - Starting async job submit for node 69c23a1f0efaf21d93107f08 2026-03-26 07:15:51,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:15:51,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23a1f0efaf21d93107f08 HTTP/1.1" 200 None 2026-03-26 07:15:51,086 - lava_callback - DEBUG - Processing node 69c23a1f0efaf21d93107f08: baseline-arm64-baylibre 2026-03-26 07:15:51,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23a1f0efaf21d93107f08 2026-03-26 07:15:51,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:15:51,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23a1f0efaf21d93107f08 HTTP/1.1" 200 None 2026-03-26 07:15:51,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:15:51,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:15:51,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:15:51,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23a1f0efaf21d93107f08 HTTP/1.1" 200 None 2026-03-26 07:15:51,155 - lava_callback - ERROR - Error processing callback for node 69c23a1f0efaf21d93107f08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42860 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:17:11,093 - lava_callback - INFO - Received callback for node 69c23a740efaf21d93107f11 INFO: 172.21.0.1:44958 - "POST /node/69c23a740efaf21d93107f11 HTTP/1.0" 202 Accepted 2026-03-26 07:17:11,099 - lava_callback - INFO - Starting async job submit for node 69c23a740efaf21d93107f11 2026-03-26 07:17:11,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:17:11,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23a740efaf21d93107f11 HTTP/1.1" 200 None 2026-03-26 07:17:11,114 - lava_callback - DEBUG - Processing node 69c23a740efaf21d93107f11: baseline-arm64-baylibre 2026-03-26 07:17:11,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23a740efaf21d93107f11 2026-03-26 07:17:11,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:17:11,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23a740efaf21d93107f11 HTTP/1.1" 200 None 2026-03-26 07:17:11,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:17:11,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:17:11,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:17:11,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23a740efaf21d93107f11 HTTP/1.1" 200 None 2026-03-26 07:17:11,344 - lava_callback - ERROR - Error processing callback for node 69c23a740efaf21d93107f11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34792 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:18:51,030 - lava_callback - INFO - Received callback for node 69c23ad00efaf21d93107f19 INFO: 172.21.0.1:56324 - "POST /node/69c23ad00efaf21d93107f19 HTTP/1.0" 202 Accepted 2026-03-26 07:18:51,036 - lava_callback - INFO - Starting async job submit for node 69c23ad00efaf21d93107f19 2026-03-26 07:18:51,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:18:51,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23ad00efaf21d93107f19 HTTP/1.1" 200 None 2026-03-26 07:18:51,052 - lava_callback - DEBUG - Processing node 69c23ad00efaf21d93107f19: baseline-arm64-baylibre 2026-03-26 07:18:51,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23ad00efaf21d93107f19 2026-03-26 07:18:51,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:18:51,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23ad00efaf21d93107f19 HTTP/1.1" 200 None 2026-03-26 07:18:51,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:18:51,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:18:51,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:18:51,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23ad00efaf21d93107f19 HTTP/1.1" 200 None 2026-03-26 07:18:51,124 - lava_callback - ERROR - Error processing callback for node 69c23ad00efaf21d93107f19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57440 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:20:11,228 - lava_callback - INFO - Received callback for node 69c23b270efaf21d93107f24 INFO: 172.21.0.1:48324 - "POST /node/69c23b270efaf21d93107f24 HTTP/1.0" 202 Accepted 2026-03-26 07:20:11,234 - lava_callback - INFO - Starting async job submit for node 69c23b270efaf21d93107f24 2026-03-26 07:20:11,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:20:11,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23b270efaf21d93107f24 HTTP/1.1" 200 None 2026-03-26 07:20:11,250 - lava_callback - DEBUG - Processing node 69c23b270efaf21d93107f24: baseline-arm64-baylibre 2026-03-26 07:20:11,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23b270efaf21d93107f24 2026-03-26 07:20:11,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:20:11,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23b270efaf21d93107f24 HTTP/1.1" 200 None 2026-03-26 07:20:11,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:20:11,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:20:11,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:20:11,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23b270efaf21d93107f24 HTTP/1.1" 200 None 2026-03-26 07:20:11,323 - lava_callback - ERROR - Error processing callback for node 69c23b270efaf21d93107f24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56340 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:21:51,194 - lava_callback - INFO - Received callback for node 69c23b7d0efaf21d93107f30 INFO: 172.21.0.1:41452 - "POST /node/69c23b7d0efaf21d93107f30 HTTP/1.0" 202 Accepted 2026-03-26 07:21:51,199 - lava_callback - INFO - Starting async job submit for node 69c23b7d0efaf21d93107f30 2026-03-26 07:21:51,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:21:51,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23b7d0efaf21d93107f30 HTTP/1.1" 200 None 2026-03-26 07:21:51,216 - lava_callback - DEBUG - Processing node 69c23b7d0efaf21d93107f30: baseline-arm64-baylibre 2026-03-26 07:21:51,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23b7d0efaf21d93107f30 2026-03-26 07:21:51,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:21:51,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23b7d0efaf21d93107f30 HTTP/1.1" 200 None 2026-03-26 07:21:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:21:51,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:21:51,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:21:51,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23b7d0efaf21d93107f30 HTTP/1.1" 200 None 2026-03-26 07:21:51,273 - lava_callback - ERROR - Error processing callback for node 69c23b7d0efaf21d93107f30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59600 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:27:35,972 - lava_callback - INFO - Received callback for node 69c23cdb0efaf21d93107f6f INFO: 172.21.0.1:59320 - "POST /node/69c23cdb0efaf21d93107f6f HTTP/1.0" 202 Accepted 2026-03-26 07:27:35,977 - lava_callback - INFO - Starting async job submit for node 69c23cdb0efaf21d93107f6f 2026-03-26 07:27:35,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:27:35,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23cdb0efaf21d93107f6f HTTP/1.1" 200 None 2026-03-26 07:27:35,993 - lava_callback - DEBUG - Processing node 69c23cdb0efaf21d93107f6f: baseline-arm64-clabbe 2026-03-26 07:27:35,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23cdb0efaf21d93107f6f 2026-03-26 07:27:35,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:27:36,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23cdb0efaf21d93107f6f HTTP/1.1" 200 None 2026-03-26 07:27:36,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:27:36,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:27:36,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:27:36,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23cdb0efaf21d93107f6f HTTP/1.1" 200 None 2026-03-26 07:27:36,096 - lava_callback - ERROR - Error processing callback for node 69c23cdb0efaf21d93107f6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55246 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:28:55,964 - lava_callback - INFO - Received callback for node 69c23d300efaf21d93107f7c INFO: 172.21.0.1:41980 - "POST /node/69c23d300efaf21d93107f7c HTTP/1.0" 202 Accepted 2026-03-26 07:28:55,969 - lava_callback - INFO - Starting async job submit for node 69c23d300efaf21d93107f7c 2026-03-26 07:28:55,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:28:55,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23d300efaf21d93107f7c HTTP/1.1" 200 None 2026-03-26 07:28:55,987 - lava_callback - DEBUG - Processing node 69c23d300efaf21d93107f7c: baseline-arm64-clabbe 2026-03-26 07:28:55,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23d300efaf21d93107f7c 2026-03-26 07:28:55,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:28:56,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23d300efaf21d93107f7c HTTP/1.1" 200 None 2026-03-26 07:28:56,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:28:56,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:28:56,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:28:56,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23d300efaf21d93107f7c HTTP/1.1" 200 None 2026-03-26 07:28:56,061 - lava_callback - ERROR - Error processing callback for node 69c23d300efaf21d93107f7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50984 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 07:30:15,920 - lava_callback - INFO - Received callback for node 69c23d850efaf21d93107f8a INFO: 172.21.0.1:51000 - "POST /node/69c23d850efaf21d93107f8a HTTP/1.0" 202 Accepted 2026-03-26 07:30:15,927 - lava_callback - INFO - Starting async job submit for node 69c23d850efaf21d93107f8a 2026-03-26 07:30:15,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:30:15,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23d850efaf21d93107f8a HTTP/1.1" 200 None 2026-03-26 07:30:15,943 - lava_callback - DEBUG - Processing node 69c23d850efaf21d93107f8a: baseline-arm64-clabbe 2026-03-26 07:30:15,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23d850efaf21d93107f8a 2026-03-26 07:30:15,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:30:15,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23d850efaf21d93107f8a HTTP/1.1" 200 None 2026-03-26 07:30:15,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:30:15,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:30:15,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:30:15,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23d850efaf21d93107f8a HTTP/1.1" 200 None 2026-03-26 07:30:15,997 - lava_callback - ERROR - Error processing callback for node 69c23d850efaf21d93107f8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41882 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:31:56,152 - lava_callback - INFO - Received callback for node 69c23dd90efaf21d93107f99 INFO: 172.21.0.1:54784 - "POST /node/69c23dd90efaf21d93107f99 HTTP/1.0" 202 Accepted 2026-03-26 07:31:56,158 - lava_callback - INFO - Starting async job submit for node 69c23dd90efaf21d93107f99 2026-03-26 07:31:56,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:31:56,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23dd90efaf21d93107f99 HTTP/1.1" 200 None 2026-03-26 07:31:56,175 - lava_callback - DEBUG - Processing node 69c23dd90efaf21d93107f99: baseline-arm64-clabbe 2026-03-26 07:31:56,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23dd90efaf21d93107f99 2026-03-26 07:31:56,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:31:56,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23dd90efaf21d93107f99 HTTP/1.1" 200 None 2026-03-26 07:31:56,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:31:56,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:31:56,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:31:56,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23dd90efaf21d93107f99 HTTP/1.1" 200 None 2026-03-26 07:31:56,228 - lava_callback - ERROR - Error processing callback for node 69c23dd90efaf21d93107f99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39934 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-26 07:33:16,183 - lava_callback - INFO - Received callback for node 69c23e2c0efaf21d93107fa5 INFO: 172.21.0.1:39936 - "POST /node/69c23e2c0efaf21d93107fa5 HTTP/1.0" 202 Accepted 2026-03-26 07:33:16,189 - lava_callback - INFO - Starting async job submit for node 69c23e2c0efaf21d93107fa5 2026-03-26 07:33:16,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:33:16,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23e2c0efaf21d93107fa5 HTTP/1.1" 200 None 2026-03-26 07:33:16,205 - lava_callback - DEBUG - Processing node 69c23e2c0efaf21d93107fa5: baseline-arm64-clabbe 2026-03-26 07:33:16,206 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23e2c0efaf21d93107fa5 2026-03-26 07:33:16,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:33:16,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23e2c0efaf21d93107fa5 HTTP/1.1" 200 None 2026-03-26 07:33:16,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:33:16,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:33:16,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:33:16,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23e2c0efaf21d93107fa5 HTTP/1.1" 200 None 2026-03-26 07:33:16,266 - lava_callback - ERROR - Error processing callback for node 69c23e2c0efaf21d93107fa5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35036 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:34:36,231 - lava_callback - INFO - Received callback for node 69c23e7e0efaf21d93107fae INFO: 172.21.0.1:49186 - "POST /node/69c23e7e0efaf21d93107fae HTTP/1.0" 202 Accepted 2026-03-26 07:34:36,236 - lava_callback - INFO - Starting async job submit for node 69c23e7e0efaf21d93107fae 2026-03-26 07:34:36,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:34:36,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23e7e0efaf21d93107fae HTTP/1.1" 200 None 2026-03-26 07:34:36,252 - lava_callback - DEBUG - Processing node 69c23e7e0efaf21d93107fae: baseline-arm64-clabbe 2026-03-26 07:34:36,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23e7e0efaf21d93107fae 2026-03-26 07:34:36,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:34:36,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23e7e0efaf21d93107fae HTTP/1.1" 200 None 2026-03-26 07:34:36,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:34:36,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-26 07:34:36,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:34:36,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23e7e0efaf21d93107fae HTTP/1.1" 200 None 2026-03-26 07:34:36,310 - lava_callback - ERROR - Error processing callback for node 69c23e7e0efaf21d93107fae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43976 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:36:52,250 - lava_callback - INFO - Received callback for node 69c23f0c0efaf21d93107fbf INFO: 172.21.0.1:46568 - "POST /node/69c23f0c0efaf21d93107fbf HTTP/1.0" 202 Accepted 2026-03-26 07:36:52,256 - lava_callback - INFO - Starting async job submit for node 69c23f0c0efaf21d93107fbf 2026-03-26 07:36:52,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:36:52,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23f0c0efaf21d93107fbf HTTP/1.1" 200 None 2026-03-26 07:36:52,303 - lava_callback - DEBUG - Processing node 69c23f0c0efaf21d93107fbf: baseline-arm64-baylibre 2026-03-26 07:36:52,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23f0c0efaf21d93107fbf 2026-03-26 07:36:52,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:36:52,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23f0c0efaf21d93107fbf HTTP/1.1" 200 None 2026-03-26 07:36:52,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:36:52,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:36:52,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:36:52,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23f0c0efaf21d93107fbf HTTP/1.1" 200 None 2026-03-26 07:36:52,374 - lava_callback - ERROR - Error processing callback for node 69c23f0c0efaf21d93107fbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33714 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:38:12,035 - lava_callback - INFO - Received callback for node 69c23f5f0efaf21d93107fc9 INFO: 172.21.0.1:46754 - "POST /node/69c23f5f0efaf21d93107fc9 HTTP/1.0" 202 Accepted 2026-03-26 07:38:12,041 - lava_callback - INFO - Starting async job submit for node 69c23f5f0efaf21d93107fc9 2026-03-26 07:38:12,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:38:12,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23f5f0efaf21d93107fc9 HTTP/1.1" 200 None 2026-03-26 07:38:12,072 - lava_callback - DEBUG - Processing node 69c23f5f0efaf21d93107fc9: baseline-arm64-baylibre 2026-03-26 07:38:12,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23f5f0efaf21d93107fc9 2026-03-26 07:38:12,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:38:12,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23f5f0efaf21d93107fc9 HTTP/1.1" 200 None 2026-03-26 07:38:12,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:38:12,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:38:12,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:38:12,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23f5f0efaf21d93107fc9 HTTP/1.1" 200 None 2026-03-26 07:38:12,126 - lava_callback - ERROR - Error processing callback for node 69c23f5f0efaf21d93107fc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37404 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:39:52,186 - lava_callback - INFO - Received callback for node 69c23fb90efaf21d93107fd2 INFO: 172.21.0.1:48056 - "POST /node/69c23fb90efaf21d93107fd2 HTTP/1.0" 202 Accepted 2026-03-26 07:39:52,192 - lava_callback - INFO - Starting async job submit for node 69c23fb90efaf21d93107fd2 2026-03-26 07:39:52,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:39:52,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23fb90efaf21d93107fd2 HTTP/1.1" 200 None 2026-03-26 07:39:52,209 - lava_callback - DEBUG - Processing node 69c23fb90efaf21d93107fd2: baseline-arm64-baylibre 2026-03-26 07:39:52,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c23fb90efaf21d93107fd2 2026-03-26 07:39:52,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:39:52,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23fb90efaf21d93107fd2 HTTP/1.1" 200 None 2026-03-26 07:39:52,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:39:52,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:39:52,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:39:52,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23fb90efaf21d93107fd2 HTTP/1.1" 200 None 2026-03-26 07:39:52,275 - lava_callback - ERROR - Error processing callback for node 69c23fb90efaf21d93107fd2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57042 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:41:12,159 - lava_callback - INFO - Received callback for node 69c2400b0efaf21d93107fe0 INFO: 172.21.0.1:43008 - "POST /node/69c2400b0efaf21d93107fe0 HTTP/1.0" 202 Accepted 2026-03-26 07:41:12,164 - lava_callback - INFO - Starting async job submit for node 69c2400b0efaf21d93107fe0 2026-03-26 07:41:12,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:41:12,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2400b0efaf21d93107fe0 HTTP/1.1" 200 None 2026-03-26 07:41:12,202 - lava_callback - DEBUG - Processing node 69c2400b0efaf21d93107fe0: baseline-arm64-baylibre 2026-03-26 07:41:12,203 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2400b0efaf21d93107fe0 2026-03-26 07:41:12,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:41:12,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2400b0efaf21d93107fe0 HTTP/1.1" 200 None 2026-03-26 07:41:12,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:41:12,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:41:12,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:41:12,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2400b0efaf21d93107fe0 HTTP/1.1" 200 None 2026-03-26 07:41:12,260 - lava_callback - ERROR - Error processing callback for node 69c2400b0efaf21d93107fe0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:42:32,262 - lava_callback - INFO - Received callback for node 69c2405f0efaf21d93107fe7 INFO: 172.21.0.1:44316 - "POST /node/69c2405f0efaf21d93107fe7 HTTP/1.0" 202 Accepted 2026-03-26 07:42:32,268 - lava_callback - INFO - Starting async job submit for node 69c2405f0efaf21d93107fe7 2026-03-26 07:42:32,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:42:32,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2405f0efaf21d93107fe7 HTTP/1.1" 200 None 2026-03-26 07:42:32,284 - lava_callback - DEBUG - Processing node 69c2405f0efaf21d93107fe7: baseline-arm64-baylibre 2026-03-26 07:42:32,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2405f0efaf21d93107fe7 2026-03-26 07:42:32,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:42:32,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2405f0efaf21d93107fe7 HTTP/1.1" 200 None 2026-03-26 07:42:32,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:42:32,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:42:32,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:42:32,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2405f0efaf21d93107fe7 HTTP/1.1" 200 None 2026-03-26 07:42:32,349 - lava_callback - ERROR - Error processing callback for node 69c2405f0efaf21d93107fe7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32884 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:43:56,619 - lava_callback - INFO - Received callback for node 69c240b90efaf21d93107ff0 INFO: 172.21.0.1:32900 - "POST /node/69c240b90efaf21d93107ff0 HTTP/1.0" 202 Accepted 2026-03-26 07:43:56,625 - lava_callback - INFO - Starting async job submit for node 69c240b90efaf21d93107ff0 2026-03-26 07:43:56,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:43:56,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c240b90efaf21d93107ff0 HTTP/1.1" 200 None 2026-03-26 07:43:56,642 - lava_callback - DEBUG - Processing node 69c240b90efaf21d93107ff0: baseline-arm64-clabbe 2026-03-26 07:43:56,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c240b90efaf21d93107ff0 2026-03-26 07:43:56,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:43:56,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c240b90efaf21d93107ff0 HTTP/1.1" 200 None 2026-03-26 07:43:56,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:43:56,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:43:56,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:43:56,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c240b90efaf21d93107ff0 HTTP/1.1" 200 None 2026-03-26 07:43:56,703 - lava_callback - ERROR - Error processing callback for node 69c240b90efaf21d93107ff0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59236 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:45:36,822 - lava_callback - INFO - Received callback for node 69c2410e0efaf21d93108000 INFO: 172.21.0.1:59248 - "POST /node/69c2410e0efaf21d93108000 HTTP/1.0" 202 Accepted 2026-03-26 07:45:36,828 - lava_callback - INFO - Starting async job submit for node 69c2410e0efaf21d93108000 2026-03-26 07:45:36,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:45:36,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2410e0efaf21d93108000 HTTP/1.1" 200 None 2026-03-26 07:45:36,844 - lava_callback - DEBUG - Processing node 69c2410e0efaf21d93108000: baseline-arm64-clabbe 2026-03-26 07:45:36,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2410e0efaf21d93108000 2026-03-26 07:45:36,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:45:36,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2410e0efaf21d93108000 HTTP/1.1" 200 None 2026-03-26 07:45:36,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:45:36,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:45:36,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:45:36,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2410e0efaf21d93108000 HTTP/1.1" 200 None 2026-03-26 07:45:36,920 - lava_callback - ERROR - Error processing callback for node 69c2410e0efaf21d93108000: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55658 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:46:56,786 - lava_callback - INFO - Received callback for node 69c241630efaf21d93108016 INFO: 172.21.0.1:55664 - "POST /node/69c241630efaf21d93108016 HTTP/1.0" 202 Accepted 2026-03-26 07:46:56,792 - lava_callback - INFO - Starting async job submit for node 69c241630efaf21d93108016 2026-03-26 07:46:56,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:46:56,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c241630efaf21d93108016 HTTP/1.1" 200 None 2026-03-26 07:46:56,807 - lava_callback - DEBUG - Processing node 69c241630efaf21d93108016: baseline-arm64-clabbe 2026-03-26 07:46:56,808 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c241630efaf21d93108016 2026-03-26 07:46:56,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:46:56,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c241630efaf21d93108016 HTTP/1.1" 200 None 2026-03-26 07:46:56,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:46:56,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:46:56,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:46:56,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c241630efaf21d93108016 HTTP/1.1" 200 None 2026-03-26 07:46:56,894 - lava_callback - ERROR - Error processing callback for node 69c241630efaf21d93108016: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46528 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:48:16,833 - lava_callback - INFO - Received callback for node 69c241b60efaf21d93108026 INFO: 172.21.0.1:46530 - "POST /node/69c241b60efaf21d93108026 HTTP/1.0" 202 Accepted 2026-03-26 07:48:16,839 - lava_callback - INFO - Starting async job submit for node 69c241b60efaf21d93108026 2026-03-26 07:48:16,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:48:16,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c241b60efaf21d93108026 HTTP/1.1" 200 None 2026-03-26 07:48:16,855 - lava_callback - DEBUG - Processing node 69c241b60efaf21d93108026: baseline-arm64-clabbe 2026-03-26 07:48:16,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c241b60efaf21d93108026 2026-03-26 07:48:16,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:48:16,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c241b60efaf21d93108026 HTTP/1.1" 200 None 2026-03-26 07:48:16,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:48:16,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:48:16,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:48:16,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c241b60efaf21d93108026 HTTP/1.1" 200 None 2026-03-26 07:48:16,912 - lava_callback - ERROR - Error processing callback for node 69c241b60efaf21d93108026: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47482 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:49:36,941 - lava_callback - INFO - Received callback for node 69c242090efaf21d93108033 INFO: 172.21.0.1:47492 - "POST /node/69c242090efaf21d93108033 HTTP/1.0" 202 Accepted 2026-03-26 07:49:36,947 - lava_callback - INFO - Starting async job submit for node 69c242090efaf21d93108033 2026-03-26 07:49:36,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:49:36,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c242090efaf21d93108033 HTTP/1.1" 200 None 2026-03-26 07:49:36,964 - lava_callback - DEBUG - Processing node 69c242090efaf21d93108033: baseline-arm64-clabbe 2026-03-26 07:49:36,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c242090efaf21d93108033 2026-03-26 07:49:36,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:49:36,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c242090efaf21d93108033 HTTP/1.1" 200 None 2026-03-26 07:49:36,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:49:37,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:49:37,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:49:37,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c242090efaf21d93108033 HTTP/1.1" 200 None 2026-03-26 07:49:37,030 - lava_callback - ERROR - Error processing callback for node 69c242090efaf21d93108033: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35728 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:51:16,981 - lava_callback - INFO - Received callback for node 69c242670efaf21d93108040 INFO: 172.21.0.1:35732 - "POST /node/69c242670efaf21d93108040 HTTP/1.0" 202 Accepted 2026-03-26 07:51:16,987 - lava_callback - INFO - Starting async job submit for node 69c242670efaf21d93108040 2026-03-26 07:51:16,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:51:17,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c242670efaf21d93108040 HTTP/1.1" 200 None 2026-03-26 07:51:17,003 - lava_callback - DEBUG - Processing node 69c242670efaf21d93108040: baseline-arm64-clabbe 2026-03-26 07:51:17,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c242670efaf21d93108040 2026-03-26 07:51:17,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:51:17,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c242670efaf21d93108040 HTTP/1.1" 200 None 2026-03-26 07:51:17,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:51:17,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:51:17,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:51:17,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c242670efaf21d93108040 HTTP/1.1" 200 None 2026-03-26 07:51:17,056 - lava_callback - ERROR - Error processing callback for node 69c242670efaf21d93108040: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:52:57,055 - lava_callback - INFO - Received callback for node 69c242c80efaf21d9310804c INFO: 172.21.0.1:36968 - "POST /node/69c242c80efaf21d9310804c HTTP/1.0" 202 Accepted 2026-03-26 07:52:57,061 - lava_callback - INFO - Starting async job submit for node 69c242c80efaf21d9310804c 2026-03-26 07:52:57,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:52:57,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c242c80efaf21d9310804c HTTP/1.1" 200 None 2026-03-26 07:52:57,077 - lava_callback - DEBUG - Processing node 69c242c80efaf21d9310804c: baseline-arm64-clabbe 2026-03-26 07:52:57,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c242c80efaf21d9310804c 2026-03-26 07:52:57,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:52:57,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c242c80efaf21d9310804c HTTP/1.1" 200 None 2026-03-26 07:52:57,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:52:57,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:52:57,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:52:57,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c242c80efaf21d9310804c HTTP/1.1" 200 None 2026-03-26 07:52:57,150 - lava_callback - ERROR - Error processing callback for node 69c242c80efaf21d9310804c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35748 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:54:17,105 - lava_callback - INFO - Received callback for node 69c2431c0efaf21d93108058 INFO: 172.21.0.1:35754 - "POST /node/69c2431c0efaf21d93108058 HTTP/1.0" 202 Accepted 2026-03-26 07:54:17,111 - lava_callback - INFO - Starting async job submit for node 69c2431c0efaf21d93108058 2026-03-26 07:54:17,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:54:17,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2431c0efaf21d93108058 HTTP/1.1" 200 None 2026-03-26 07:54:17,127 - lava_callback - DEBUG - Processing node 69c2431c0efaf21d93108058: baseline-arm64-clabbe 2026-03-26 07:54:17,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2431c0efaf21d93108058 2026-03-26 07:54:17,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:54:17,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2431c0efaf21d93108058 HTTP/1.1" 200 None 2026-03-26 07:54:17,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:54:17,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:54:17,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:54:17,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2431c0efaf21d93108058 HTTP/1.1" 200 None 2026-03-26 07:54:17,185 - lava_callback - ERROR - Error processing callback for node 69c2431c0efaf21d93108058: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:55:37,201 - lava_callback - INFO - Received callback for node 69c243710efaf21d9310806c INFO: 172.21.0.1:38754 - "POST /node/69c243710efaf21d9310806c HTTP/1.0" 202 Accepted 2026-03-26 07:55:37,206 - lava_callback - INFO - Starting async job submit for node 69c243710efaf21d9310806c 2026-03-26 07:55:37,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:55:37,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c243710efaf21d9310806c HTTP/1.1" 200 None 2026-03-26 07:55:37,245 - lava_callback - DEBUG - Processing node 69c243710efaf21d9310806c: baseline-arm64-clabbe 2026-03-26 07:55:37,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c243710efaf21d9310806c 2026-03-26 07:55:37,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:55:37,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c243710efaf21d9310806c HTTP/1.1" 200 None 2026-03-26 07:55:37,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:55:37,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:55:37,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:55:37,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c243710efaf21d9310806c HTTP/1.1" 200 None 2026-03-26 07:55:37,311 - lava_callback - ERROR - Error processing callback for node 69c243710efaf21d9310806c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36554 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:56:57,234 - lava_callback - INFO - Received callback for node 69c243c30efaf21d9310807d INFO: 172.21.0.1:36570 - "POST /node/69c243c30efaf21d9310807d HTTP/1.0" 202 Accepted 2026-03-26 07:56:57,240 - lava_callback - INFO - Starting async job submit for node 69c243c30efaf21d9310807d 2026-03-26 07:56:57,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:56:57,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c243c30efaf21d9310807d HTTP/1.1" 200 None 2026-03-26 07:56:57,259 - lava_callback - DEBUG - Processing node 69c243c30efaf21d9310807d: baseline-arm64-clabbe 2026-03-26 07:56:57,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c243c30efaf21d9310807d 2026-03-26 07:56:57,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:56:57,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c243c30efaf21d9310807d HTTP/1.1" 200 None 2026-03-26 07:56:57,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:56:57,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:56:57,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:56:57,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c243c30efaf21d9310807d HTTP/1.1" 200 None 2026-03-26 07:56:57,321 - lava_callback - ERROR - Error processing callback for node 69c243c30efaf21d9310807d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41120 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:58:37,317 - lava_callback - INFO - Received callback for node 69c244170efaf21d93108093 INFO: 172.21.0.1:41136 - "POST /node/69c244170efaf21d93108093 HTTP/1.0" 202 Accepted 2026-03-26 07:58:37,323 - lava_callback - INFO - Starting async job submit for node 69c244170efaf21d93108093 2026-03-26 07:58:37,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:58:37,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c244170efaf21d93108093 HTTP/1.1" 200 None 2026-03-26 07:58:37,342 - lava_callback - DEBUG - Processing node 69c244170efaf21d93108093: baseline-arm64-clabbe 2026-03-26 07:58:37,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c244170efaf21d93108093 2026-03-26 07:58:37,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:58:37,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c244170efaf21d93108093 HTTP/1.1" 200 None 2026-03-26 07:58:37,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:58:37,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:58:37,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:58:37,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c244170efaf21d93108093 HTTP/1.1" 200 None 2026-03-26 07:58:37,405 - lava_callback - ERROR - Error processing callback for node 69c244170efaf21d93108093: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK 2026-03-26 07:59:57,373 - lava_callback - INFO - Received callback for node 69c2446b0efaf21d931080a9 INFO: 172.21.0.1:60044 - "POST /node/69c2446b0efaf21d931080a9 HTTP/1.0" 202 Accepted 2026-03-26 07:59:57,378 - lava_callback - INFO - Starting async job submit for node 69c2446b0efaf21d931080a9 2026-03-26 07:59:57,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:59:57,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2446b0efaf21d931080a9 HTTP/1.1" 200 None 2026-03-26 07:59:57,395 - lava_callback - DEBUG - Processing node 69c2446b0efaf21d931080a9: baseline-arm64-clabbe 2026-03-26 07:59:57,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2446b0efaf21d931080a9 2026-03-26 07:59:57,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:59:57,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2446b0efaf21d931080a9 HTTP/1.1" 200 None 2026-03-26 07:59:57,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:59:57,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-26 07:59:57,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 07:59:57,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2446b0efaf21d931080a9 HTTP/1.1" 200 None 2026-03-26 07:59:57,455 - lava_callback - ERROR - Error processing callback for node 69c2446b0efaf21d931080a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36248 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:01:13,506 - lava_callback - INFO - Received callback for node 69c244ba0efaf21d931080c0 INFO: 172.21.0.1:41840 - "POST /node/69c244ba0efaf21d931080c0 HTTP/1.0" 202 Accepted 2026-03-26 08:01:13,511 - lava_callback - INFO - Starting async job submit for node 69c244ba0efaf21d931080c0 2026-03-26 08:01:13,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:01:13,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c244ba0efaf21d931080c0 HTTP/1.1" 200 None 2026-03-26 08:01:13,528 - lava_callback - DEBUG - Processing node 69c244ba0efaf21d931080c0: baseline-arm64-baylibre 2026-03-26 08:01:13,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c244ba0efaf21d931080c0 2026-03-26 08:01:13,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:01:13,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c244ba0efaf21d931080c0 HTTP/1.1" 200 None 2026-03-26 08:01:13,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:01:13,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:01:13,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:01:13,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c244ba0efaf21d931080c0 HTTP/1.1" 200 None 2026-03-26 08:01:13,583 - lava_callback - ERROR - Error processing callback for node 69c244ba0efaf21d931080c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55086 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:02:37,500 - lava_callback - INFO - Received callback for node 69c245130efaf21d931080d7 INFO: 172.21.0.1:55094 - "POST /node/69c245130efaf21d931080d7 HTTP/1.0" 202 Accepted 2026-03-26 08:02:37,506 - lava_callback - INFO - Starting async job submit for node 69c245130efaf21d931080d7 2026-03-26 08:02:37,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:02:37,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c245130efaf21d931080d7 HTTP/1.1" 200 None 2026-03-26 08:02:37,545 - lava_callback - DEBUG - Processing node 69c245130efaf21d931080d7: baseline-arm64-clabbe 2026-03-26 08:02:37,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c245130efaf21d931080d7 2026-03-26 08:02:37,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:02:37,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c245130efaf21d931080d7 HTTP/1.1" 200 None 2026-03-26 08:02:37,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:02:37,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:02:37,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:02:37,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c245130efaf21d931080d7 HTTP/1.1" 200 None 2026-03-26 08:02:37,605 - lava_callback - ERROR - Error processing callback for node 69c245130efaf21d931080d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58456 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:04:17,577 - lava_callback - INFO - Received callback for node 69c2456b0efaf21d931080e5 INFO: 172.21.0.1:58470 - "POST /node/69c2456b0efaf21d931080e5 HTTP/1.0" 202 Accepted 2026-03-26 08:04:17,583 - lava_callback - INFO - Starting async job submit for node 69c2456b0efaf21d931080e5 2026-03-26 08:04:17,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:04:17,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2456b0efaf21d931080e5 HTTP/1.1" 200 None 2026-03-26 08:04:17,608 - lava_callback - DEBUG - Processing node 69c2456b0efaf21d931080e5: baseline-arm64-clabbe 2026-03-26 08:04:17,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2456b0efaf21d931080e5 2026-03-26 08:04:17,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:04:17,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2456b0efaf21d931080e5 HTTP/1.1" 200 None 2026-03-26 08:04:17,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:04:17,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:04:17,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:04:17,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2456b0efaf21d931080e5 HTTP/1.1" 200 None 2026-03-26 08:04:17,663 - lava_callback - ERROR - Error processing callback for node 69c2456b0efaf21d931080e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37866 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46538 - "GET /security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:05:37,592 - lava_callback - INFO - Received callback for node 69c245c00efaf21d931080f1 INFO: 172.21.0.1:33824 - "POST /node/69c245c00efaf21d931080f1 HTTP/1.0" 202 Accepted 2026-03-26 08:05:37,597 - lava_callback - INFO - Starting async job submit for node 69c245c00efaf21d931080f1 2026-03-26 08:05:37,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:05:37,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c245c00efaf21d931080f1 HTTP/1.1" 200 None 2026-03-26 08:05:37,614 - lava_callback - DEBUG - Processing node 69c245c00efaf21d931080f1: baseline-arm64-clabbe 2026-03-26 08:05:37,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c245c00efaf21d931080f1 2026-03-26 08:05:37,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:05:37,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c245c00efaf21d931080f1 HTTP/1.1" 200 None 2026-03-26 08:05:37,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:05:37,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:05:37,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:05:37,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c245c00efaf21d931080f1 HTTP/1.1" 200 None 2026-03-26 08:05:37,670 - lava_callback - ERROR - Error processing callback for node 69c245c00efaf21d931080f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53638 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:06:57,608 - lava_callback - INFO - Received callback for node 69c246150efaf21d931080fc INFO: 172.21.0.1:53650 - "POST /node/69c246150efaf21d931080fc HTTP/1.0" 202 Accepted 2026-03-26 08:06:57,614 - lava_callback - INFO - Starting async job submit for node 69c246150efaf21d931080fc 2026-03-26 08:06:57,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:06:57,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c246150efaf21d931080fc HTTP/1.1" 200 None 2026-03-26 08:06:57,634 - lava_callback - DEBUG - Processing node 69c246150efaf21d931080fc: baseline-arm64-clabbe 2026-03-26 08:06:57,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c246150efaf21d931080fc 2026-03-26 08:06:57,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:06:57,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c246150efaf21d931080fc HTTP/1.1" 200 None 2026-03-26 08:06:57,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:06:57,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:06:57,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:06:57,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c246150efaf21d931080fc HTTP/1.1" 200 None 2026-03-26 08:06:57,690 - lava_callback - ERROR - Error processing callback for node 69c246150efaf21d931080fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37464 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:08:17,661 - lava_callback - INFO - Received callback for node 69c246670efaf21d93108109 INFO: 172.21.0.1:37480 - "POST /node/69c246670efaf21d93108109 HTTP/1.0" 202 Accepted 2026-03-26 08:08:17,667 - lava_callback - INFO - Starting async job submit for node 69c246670efaf21d93108109 2026-03-26 08:08:17,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:08:17,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c246670efaf21d93108109 HTTP/1.1" 200 None 2026-03-26 08:08:17,686 - lava_callback - DEBUG - Processing node 69c246670efaf21d93108109: baseline-arm64-clabbe 2026-03-26 08:08:17,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c246670efaf21d93108109 2026-03-26 08:08:17,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:08:17,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c246670efaf21d93108109 HTTP/1.1" 200 None 2026-03-26 08:08:17,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:08:17,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:08:17,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:08:17,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c246670efaf21d93108109 HTTP/1.1" 200 None 2026-03-26 08:08:17,747 - lava_callback - ERROR - Error processing callback for node 69c246670efaf21d93108109: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:09:37,837 - lava_callback - INFO - Received callback for node 69c246bc0efaf21d93108115 INFO: 172.21.0.1:47332 - "POST /node/69c246bc0efaf21d93108115 HTTP/1.0" 202 Accepted 2026-03-26 08:09:37,843 - lava_callback - INFO - Starting async job submit for node 69c246bc0efaf21d93108115 2026-03-26 08:09:37,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:09:37,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c246bc0efaf21d93108115 HTTP/1.1" 200 None 2026-03-26 08:09:37,859 - lava_callback - DEBUG - Processing node 69c246bc0efaf21d93108115: baseline-arm64-clabbe 2026-03-26 08:09:37,860 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c246bc0efaf21d93108115 2026-03-26 08:09:37,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:09:37,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c246bc0efaf21d93108115 HTTP/1.1" 200 None 2026-03-26 08:09:37,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:09:37,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:09:37,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:09:37,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c246bc0efaf21d93108115 HTTP/1.1" 200 None 2026-03-26 08:09:37,918 - lava_callback - ERROR - Error processing callback for node 69c246bc0efaf21d93108115: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36222 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:10:53,868 - lava_callback - INFO - Received callback for node 69c2470a0efaf21d9310811a INFO: 172.21.0.1:45368 - "POST /node/69c2470a0efaf21d9310811a HTTP/1.0" 202 Accepted 2026-03-26 08:10:53,874 - lava_callback - INFO - Starting async job submit for node 69c2470a0efaf21d9310811a 2026-03-26 08:10:53,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:10:53,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2470a0efaf21d9310811a HTTP/1.1" 200 None 2026-03-26 08:10:53,890 - lava_callback - DEBUG - Processing node 69c2470a0efaf21d9310811a: baseline-arm64-baylibre 2026-03-26 08:10:53,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c2470a0efaf21d9310811a 2026-03-26 08:10:53,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:10:53,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2470a0efaf21d9310811a HTTP/1.1" 200 None 2026-03-26 08:10:53,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:10:53,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:10:53,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:10:53,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2470a0efaf21d9310811a HTTP/1.1" 200 None 2026-03-26 08:10:53,970 - lava_callback - ERROR - Error processing callback for node 69c2470a0efaf21d9310811a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46546 - "GET / HTTP/1.0" 200 OK 2026-03-26 08:12:37,939 - lava_callback - INFO - Received callback for node 69c247630efaf21d9310812e INFO: 172.21.0.1:46562 - "POST /node/69c247630efaf21d9310812e HTTP/1.0" 202 Accepted 2026-03-26 08:12:37,946 - lava_callback - INFO - Starting async job submit for node 69c247630efaf21d9310812e 2026-03-26 08:12:37,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:12:37,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c247630efaf21d9310812e HTTP/1.1" 200 None 2026-03-26 08:12:37,962 - lava_callback - DEBUG - Processing node 69c247630efaf21d9310812e: baseline-arm64-clabbe 2026-03-26 08:12:37,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c247630efaf21d9310812e 2026-03-26 08:12:37,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:12:37,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c247630efaf21d9310812e HTTP/1.1" 200 None 2026-03-26 08:12:37,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:12:38,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-26 08:12:38,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-26 08:12:38,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c247630efaf21d9310812e HTTP/1.1" 200 None 2026-03-26 08:12:38,026 - lava_callback - ERROR - Error processing callback for node 69c247630efaf21d9310812e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]