INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50420 - "GET /login HTTP/1.0" 404 Not Found INFO: 172.20.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:33:33,607 - lava_callback - INFO - Received callback for node 69d6126b4b6493d861662001 2026-04-08 08:33:33,615 - lava_callback - INFO - Starting async job submit for node 69d6126b4b6493d861662001 INFO: 172.20.0.1:48784 - "POST /node/69d6126b4b6493d861662001 HTTP/1.0" 202 Accepted 2026-04-08 08:33:33,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:33:33,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6126b4b6493d861662001 HTTP/1.1" 200 None 2026-04-08 08:33:33,667 - lava_callback - DEBUG - Processing node 69d6126b4b6493d861662001: baseline-arm 2026-04-08 08:33:33,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:33:33,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:33:33,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d6126b4b6493d861662001/log.txt.gz 2026-04-08 08:33:34,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:33:34,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:33:34,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d6126b4b6493d861662001/lava-logs.yaml 2026-04-08 08:33:34,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:33:34,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:33:34,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d6126b4b6493d861662001/lava_callback.json.gz 2026-04-08 08:33:34,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:33:34,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6126b4b6493d861662001 HTTP/1.1" 200 None 2026-04-08 08:33:34,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:33:34,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f134b6493d861661fa6 HTTP/1.1" 200 None 2026-04-08 08:33:34,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:33:34,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6126b4b6493d861662001 HTTP/1.1" 200 None 2026-04-08 08:33:34,554 - lava_callback - INFO - Completed processing callback for node 69d6126b4b6493d861662001 INFO: 172.20.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44304 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:34:04,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:34:04,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40900 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:38:51,489 - lava_callback - INFO - Received callback for node 69d3710e2214054febe37ff9 INFO: 172.20.0.1:59646 - "POST /node/69d3710e2214054febe37ff9 HTTP/1.0" 202 Accepted 2026-04-08 08:38:51,495 - lava_callback - INFO - Starting async job submit for node 69d3710e2214054febe37ff9 2026-04-08 08:38:51,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:38:51,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3710e2214054febe37ff9 HTTP/1.1" 200 None 2026-04-08 08:38:51,515 - lava_callback - DEBUG - Processing node 69d3710e2214054febe37ff9: baseline-arm-baylibre 2026-04-08 08:38:51,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3710e2214054febe37ff9 2026-04-08 08:38:51,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:38:51,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3710e2214054febe37ff9 HTTP/1.1" 200 None 2026-04-08 08:38:51,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:38:51,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:38:51,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:38:51,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3710e2214054febe37ff9 HTTP/1.1" 200 None 2026-04-08 08:38:51,598 - lava_callback - ERROR - Error processing callback for node 69d3710e2214054febe37ff9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39190 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:39:11,385 - lava_callback - INFO - Received callback for node 69d3712a2214054febe38029 INFO: 172.20.0.1:40334 - "POST /node/69d3712a2214054febe38029 HTTP/1.0" 202 Accepted 2026-04-08 08:39:11,393 - lava_callback - INFO - Starting async job submit for node 69d3712a2214054febe38029 2026-04-08 08:39:11,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:11,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3712a2214054febe38029 HTTP/1.1" 200 None 2026-04-08 08:39:11,421 - lava_callback - DEBUG - Processing node 69d3712a2214054febe38029: baseline-arm-baylibre 2026-04-08 08:39:11,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3712a2214054febe38029 2026-04-08 08:39:11,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:11,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3712a2214054febe38029 HTTP/1.1" 200 None 2026-04-08 08:39:11,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:11,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:39:11,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:11,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3712a2214054febe38029 HTTP/1.1" 200 None 2026-04-08 08:39:11,505 - lava_callback - ERROR - Error processing callback for node 69d3712a2214054febe38029: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59602 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:39:51,457 - lava_callback - INFO - Received callback for node 69d371482214054febe3806e INFO: 172.20.0.1:48448 - "POST /node/69d371482214054febe3806e HTTP/1.0" 202 Accepted 2026-04-08 08:39:51,464 - lava_callback - INFO - Starting async job submit for node 69d371482214054febe3806e 2026-04-08 08:39:51,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:51,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371482214054febe3806e HTTP/1.1" 200 None 2026-04-08 08:39:51,495 - lava_callback - DEBUG - Processing node 69d371482214054febe3806e: baseline-arm-baylibre 2026-04-08 08:39:51,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d371482214054febe3806e 2026-04-08 08:39:51,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:51,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371482214054febe3806e HTTP/1.1" 200 None 2026-04-08 08:39:51,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:51,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:39:51,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:39:51,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d371482214054febe3806e HTTP/1.1" 200 None 2026-04-08 08:39:51,588 - lava_callback - ERROR - Error processing callback for node 69d371482214054febe3806e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46074 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:40:11,461 - lava_callback - INFO - Received callback for node 69d371682214054febe38099 INFO: 172.20.0.1:45940 - "POST /node/69d371682214054febe38099 HTTP/1.0" 202 Accepted 2026-04-08 08:40:11,466 - lava_callback - INFO - Starting async job submit for node 69d371682214054febe38099 2026-04-08 08:40:11,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:11,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371682214054febe38099 HTTP/1.1" 200 None 2026-04-08 08:40:11,482 - lava_callback - DEBUG - Processing node 69d371682214054febe38099: baseline-arm-baylibre 2026-04-08 08:40:11,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d371682214054febe38099 2026-04-08 08:40:11,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:11,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371682214054febe38099 HTTP/1.1" 200 None 2026-04-08 08:40:11,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:11,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:40:11,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:11,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d371682214054febe38099 HTTP/1.1" 200 None 2026-04-08 08:40:11,538 - lava_callback - ERROR - Error processing callback for node 69d371682214054febe38099: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49648 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:40:50,273 - lava_callback - INFO - Received callback for node 69d3718c2214054febe380b2 INFO: 172.20.0.1:40500 - "POST /node/69d3718c2214054febe380b2 HTTP/1.0" 202 Accepted 2026-04-08 08:40:50,279 - lava_callback - INFO - Starting async job submit for node 69d3718c2214054febe380b2 2026-04-08 08:40:50,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:50,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3718c2214054febe380b2 HTTP/1.1" 200 None 2026-04-08 08:40:50,295 - lava_callback - DEBUG - Processing node 69d3718c2214054febe380b2: baseline-arm-clabbe 2026-04-08 08:40:50,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3718c2214054febe380b2 2026-04-08 08:40:50,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:50,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3718c2214054febe380b2 HTTP/1.1" 200 None 2026-04-08 08:40:50,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:50,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:40:50,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:40:50,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3718c2214054febe380b2 HTTP/1.1" 200 None 2026-04-08 08:40:50,361 - lava_callback - ERROR - Error processing callback for node 69d3718c2214054febe380b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:41:27,969 - lava_callback - INFO - Received callback for node 69d614a44b6493d861662106 INFO: 172.20.0.1:56314 - "POST /node/69d614a44b6493d861662106 HTTP/1.0" 202 Accepted 2026-04-08 08:41:27,975 - lava_callback - INFO - Starting async job submit for node 69d614a44b6493d861662106 2026-04-08 08:41:27,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:28,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a44b6493d861662106 HTTP/1.1" 200 None 2026-04-08 08:41:28,011 - lava_callback - DEBUG - Processing node 69d614a44b6493d861662106: baseline-arm 2026-04-08 08:41:28,016 - lava_callback - WARNING - Failed to upload log 2026-04-08 08:41:28,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:28,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:28,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a44b6493d861662106/lava-logs.yaml 2026-04-08 08:41:28,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:28,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:28,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a44b6493d861662106/lava_callback.json.gz 2026-04-08 08:41:28,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:28,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a44b6493d861662106 HTTP/1.1" 200 None 2026-04-08 08:41:28,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:28,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:41:28,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:28,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614a44b6493d861662106 HTTP/1.1" 200 None 2026-04-08 08:41:28,524 - lava_callback - INFO - Completed processing callback for node 69d614a44b6493d861662106 2026-04-08 08:41:30,259 - lava_callback - INFO - Received callback for node 69d371b12214054febe380bd INFO: 172.20.0.1:33844 - "POST /node/69d371b12214054febe380bd HTTP/1.0" 202 Accepted 2026-04-08 08:41:30,269 - lava_callback - INFO - Starting async job submit for node 69d371b12214054febe380bd 2026-04-08 08:41:30,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:30,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371b12214054febe380bd HTTP/1.1" 200 None 2026-04-08 08:41:30,290 - lava_callback - DEBUG - Processing node 69d371b12214054febe380bd: baseline-arm-clabbe 2026-04-08 08:41:30,290 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d371b12214054febe380bd 2026-04-08 08:41:30,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:30,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371b12214054febe380bd HTTP/1.1" 200 None 2026-04-08 08:41:30,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:30,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:41:30,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:30,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d371b12214054febe380bd HTTP/1.1" 200 None 2026-04-08 08:41:30,345 - lava_callback - ERROR - Error processing callback for node 69d371b12214054febe380bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:41:31,998 - lava_callback - INFO - Received callback for node 69d6149c4b6493d8616620bc INFO: 172.20.0.1:33854 - "POST /node/69d6149c4b6493d8616620bc HTTP/1.0" 202 Accepted 2026-04-08 08:41:32,006 - lava_callback - INFO - Starting async job submit for node 69d6149c4b6493d8616620bc 2026-04-08 08:41:32,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:32,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149c4b6493d8616620bc HTTP/1.1" 200 None 2026-04-08 08:41:32,040 - lava_callback - DEBUG - Processing node 69d6149c4b6493d8616620bc: baseline-x86 2026-04-08 08:41:32,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:32,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:32,470 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149c4b6493d8616620bc/log.txt.gz 2026-04-08 08:41:32,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:32,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:32,745 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149c4b6493d8616620bc/lava-logs.yaml 2026-04-08 08:41:32,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:32,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:32,896 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149c4b6493d8616620bc/lava_callback.json.gz 2026-04-08 08:41:32,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:32,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149c4b6493d8616620bc HTTP/1.1" 200 None 2026-04-08 08:41:32,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:33,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f334b6493d861661fc9 HTTP/1.1" 200 None 2026-04-08 08:41:33,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:33,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6149c4b6493d8616620bc HTTP/1.1" 200 None 2026-04-08 08:41:33,113 - lava_callback - INFO - Completed processing callback for node 69d6149c4b6493d8616620bc 2026-04-08 08:41:34,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:34,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33870 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:41:41,032 - lava_callback - INFO - Received callback for node 69d614a44b6493d861662109 INFO: 172.20.0.1:45622 - "POST /node/69d614a44b6493d861662109 HTTP/1.0" 202 Accepted 2026-04-08 08:41:41,083 - lava_callback - INFO - Starting async job submit for node 69d614a44b6493d861662109 2026-04-08 08:41:41,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:41,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a44b6493d861662109 HTTP/1.1" 200 None 2026-04-08 08:41:41,127 - lava_callback - DEBUG - Processing node 69d614a44b6493d861662109: baseline-arm 2026-04-08 08:41:41,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:41,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:41,362 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a44b6493d861662109/log.txt.gz 2026-04-08 08:41:41,423 - lava_callback - INFO - Received callback for node 69d6149a4b6493d8616620b3 2026-04-08 08:41:41,439 - lava_callback - INFO - Starting async job submit for node 69d6149a4b6493d8616620b3 INFO: 172.20.0.1:45628 - "POST /node/69d6149a4b6493d8616620b3 HTTP/1.0" 202 Accepted 2026-04-08 08:41:41,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:41,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:41,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149a4b6493d8616620b3 HTTP/1.1" 200 None 2026-04-08 08:41:41,488 - lava_callback - DEBUG - Processing node 69d6149a4b6493d8616620b3: baseline-x86 2026-04-08 08:41:41,653 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:41,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a44b6493d861662109/lava-logs.yaml 2026-04-08 08:41:41,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:41,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:41,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:41,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a44b6493d861662109/lava_callback.json.gz 2026-04-08 08:41:41,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:41,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:41,973 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149a4b6493d8616620b3/log.txt.gz 2026-04-08 08:41:42,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a44b6493d861662109 HTTP/1.1" 200 None 2026-04-08 08:41:42,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:42,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:42,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:41:42,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:42,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614a44b6493d861662109 HTTP/1.1" 200 None 2026-04-08 08:41:42,188 - lava_callback - INFO - Completed processing callback for node 69d614a44b6493d861662109 2026-04-08 08:41:42,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:42,403 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149a4b6493d8616620b3/lava-logs.yaml 2026-04-08 08:41:42,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:42,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:42,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149a4b6493d8616620b3/lava_callback.json.gz 2026-04-08 08:41:42,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:42,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149a4b6493d8616620b3 HTTP/1.1" 200 None 2026-04-08 08:41:42,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:42,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f334b6493d861661fc9 HTTP/1.1" 200 None 2026-04-08 08:41:42,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:42,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6149a4b6493d8616620b3 HTTP/1.1" 200 None 2026-04-08 08:41:42,765 - lava_callback - INFO - Completed processing callback for node 69d6149a4b6493d8616620b3 INFO: 172.20.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36166 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:41:58,787 - lava_callback - INFO - Received callback for node 69d614a74b6493d861662112 INFO: 172.20.0.1:36170 - "POST /node/69d614a74b6493d861662112 HTTP/1.0" 202 Accepted 2026-04-08 08:41:58,834 - lava_callback - INFO - Starting async job submit for node 69d614a74b6493d861662112 2026-04-08 08:41:58,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:58,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a74b6493d861662112 HTTP/1.1" 200 None 2026-04-08 08:41:58,869 - lava_callback - DEBUG - Processing node 69d614a74b6493d861662112: baseline-arm 2026-04-08 08:41:58,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:59,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:59,133 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a74b6493d861662112/log.txt.gz 2026-04-08 08:41:59,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:59,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:59,366 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a74b6493d861662112/lava-logs.yaml 2026-04-08 08:41:59,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:41:59,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:41:59,485 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a74b6493d861662112/lava_callback.json.gz 2026-04-08 08:41:59,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:59,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a74b6493d861662112 HTTP/1.1" 200 None 2026-04-08 08:41:59,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:59,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:41:59,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:41:59,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614a74b6493d861662112 HTTP/1.1" 200 None 2026-04-08 08:41:59,559 - lava_callback - INFO - Completed processing callback for node 69d614a74b6493d861662112 2026-04-08 08:42:04,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:04,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46404 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:42:10,269 - lava_callback - INFO - Received callback for node 69d6149b4b6493d8616620b6 INFO: 172.20.0.1:54902 - "POST /node/69d6149b4b6493d8616620b6 HTTP/1.0" 202 Accepted 2026-04-08 08:42:10,275 - lava_callback - INFO - Starting async job submit for node 69d6149b4b6493d8616620b6 2026-04-08 08:42:10,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:10,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149b4b6493d8616620b6 HTTP/1.1" 200 None 2026-04-08 08:42:10,311 - lava_callback - DEBUG - Processing node 69d6149b4b6493d8616620b6: baseline-x86 2026-04-08 08:42:10,350 - lava_callback - INFO - Received callback for node 69d371d52214054febe380e8 INFO: 172.20.0.1:54918 - "POST /node/69d371d52214054febe380e8 HTTP/1.0" 202 Accepted 2026-04-08 08:42:10,371 - lava_callback - INFO - Starting async job submit for node 69d371d52214054febe380e8 2026-04-08 08:42:10,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:10,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371d52214054febe380e8 HTTP/1.1" 200 None 2026-04-08 08:42:10,472 - lava_callback - DEBUG - Processing node 69d371d52214054febe380e8: baseline-arm-clabbe 2026-04-08 08:42:10,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d371d52214054febe380e8 2026-04-08 08:42:10,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:10,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371d52214054febe380e8 HTTP/1.1" 200 None 2026-04-08 08:42:10,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:10,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:10,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:42:10,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:10,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d371d52214054febe380e8 HTTP/1.1" 200 None 2026-04-08 08:42:10,646 - lava_callback - ERROR - Error processing callback for node 69d371d52214054febe380e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:42:10,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:10,824 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149b4b6493d8616620b6/log.txt.gz 2026-04-08 08:42:11,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:11,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:11,244 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149b4b6493d8616620b6/lava-logs.yaml 2026-04-08 08:42:11,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:11,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:11,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149b4b6493d8616620b6/lava_callback.json.gz 2026-04-08 08:42:11,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:11,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149b4b6493d8616620b6 HTTP/1.1" 200 None 2026-04-08 08:42:11,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:11,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f334b6493d861661fc9 HTTP/1.1" 200 None 2026-04-08 08:42:11,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:11,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6149b4b6493d8616620b6 HTTP/1.1" 200 None 2026-04-08 08:42:11,575 - lava_callback - INFO - Completed processing callback for node 69d6149b4b6493d8616620b6 INFO: 172.20.0.1:54928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54930 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:42:24,913 - lava_callback - INFO - Received callback for node 69d614a34b6493d861662103 INFO: 172.20.0.1:60514 - "POST /node/69d614a34b6493d861662103 HTTP/1.0" 202 Accepted 2026-04-08 08:42:24,919 - lava_callback - INFO - Starting async job submit for node 69d614a34b6493d861662103 2026-04-08 08:42:24,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:24,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a34b6493d861662103 HTTP/1.1" 200 None 2026-04-08 08:42:24,956 - lava_callback - DEBUG - Processing node 69d614a34b6493d861662103: baseline-arm 2026-04-08 08:42:25,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:25,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:25,303 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a34b6493d861662103/log.txt.gz 2026-04-08 08:42:25,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:25,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:25,601 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a34b6493d861662103/lava-logs.yaml 2026-04-08 08:42:25,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:25,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:25,788 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a34b6493d861662103/lava_callback.json.gz 2026-04-08 08:42:25,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:25,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a34b6493d861662103 HTTP/1.1" 200 None 2026-04-08 08:42:25,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:25,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:42:25,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:25,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614a34b6493d861662103 HTTP/1.1" 200 None 2026-04-08 08:42:25,870 - lava_callback - INFO - Completed processing callback for node 69d614a34b6493d861662103 INFO: 172.20.0.1:60524 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:42:33,928 - lava_callback - INFO - Received callback for node 69d614b44b6493d861662133 INFO: 172.20.0.1:33348 - "POST /node/69d614b44b6493d861662133 HTTP/1.0" 202 Accepted 2026-04-08 08:42:33,934 - lava_callback - INFO - Starting async job submit for node 69d614b44b6493d861662133 2026-04-08 08:42:33,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:33,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614b44b6493d861662133 HTTP/1.1" 200 None 2026-04-08 08:42:33,973 - lava_callback - DEBUG - Processing node 69d614b44b6493d861662133: baseline-arm-broonie 2026-04-08 08:42:34,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:34,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:34,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b44b6493d861662133/log.txt.gz 2026-04-08 08:42:34,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:34,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:34,651 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b44b6493d861662133/lava-logs.yaml 2026-04-08 08:42:34,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:34,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:34,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-08 08:42:34,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:34,786 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b44b6493d861662133/lava_callback.json.gz 2026-04-08 08:42:34,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:34,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614b44b6493d861662133 HTTP/1.1" 200 None 2026-04-08 08:42:34,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:34,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:42:34,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:34,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614b44b6493d861662133 HTTP/1.1" 200 None 2026-04-08 08:42:34,884 - lava_callback - INFO - Completed processing callback for node 69d614b44b6493d861662133 2026-04-08 08:42:35,466 - lava_callback - INFO - Received callback for node 69d614b64b6493d861662136 INFO: 172.20.0.1:33352 - "POST /node/69d614b64b6493d861662136 HTTP/1.0" 202 Accepted 2026-04-08 08:42:35,479 - lava_callback - INFO - Starting async job submit for node 69d614b64b6493d861662136 2026-04-08 08:42:35,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:35,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614b64b6493d861662136 HTTP/1.1" 200 None 2026-04-08 08:42:35,524 - lava_callback - DEBUG - Processing node 69d614b64b6493d861662136: baseline-arm-broonie 2026-04-08 08:42:35,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:36,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:36,123 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b64b6493d861662136/log.txt.gz 2026-04-08 08:42:36,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:36,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:36,417 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b64b6493d861662136/lava-logs.yaml 2026-04-08 08:42:36,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33366 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:42:36,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:36,573 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b64b6493d861662136/lava_callback.json.gz 2026-04-08 08:42:36,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:36,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614b64b6493d861662136 HTTP/1.1" 200 None 2026-04-08 08:42:36,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:36,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:42:36,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:36,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614b64b6493d861662136 HTTP/1.1" 200 None 2026-04-08 08:42:36,658 - lava_callback - INFO - Completed processing callback for node 69d614b64b6493d861662136 2026-04-08 08:42:38,474 - lava_callback - INFO - Received callback for node 69d614c94b6493d8616621b8 INFO: 172.20.0.1:33368 - "POST /node/69d614c94b6493d8616621b8 HTTP/1.0" 202 Accepted 2026-04-08 08:42:38,521 - lava_callback - INFO - Starting async job submit for node 69d614c94b6493d8616621b8 2026-04-08 08:42:38,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:38,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c94b6493d8616621b8 HTTP/1.1" 200 None 2026-04-08 08:42:38,562 - lava_callback - DEBUG - Processing node 69d614c94b6493d8616621b8: baseline-arm-pengutronix 2026-04-08 08:42:38,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:38,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:38,782 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c94b6493d8616621b8/log.txt.gz 2026-04-08 08:42:38,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:42:39,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:39,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c94b6493d8616621b8/lava-logs.yaml 2026-04-08 08:42:39,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:39,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:39,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c94b6493d8616621b8/lava_callback.json.gz 2026-04-08 08:42:39,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:39,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c94b6493d8616621b8 HTTP/1.1" 200 None 2026-04-08 08:42:39,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:39,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:42:39,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:39,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614c94b6493d8616621b8 HTTP/1.1" 200 None 2026-04-08 08:42:39,213 - lava_callback - INFO - Completed processing callback for node 69d614c94b6493d8616621b8 2026-04-08 08:42:42,395 - lava_callback - INFO - Received callback for node 69d614c74b6493d8616621b2 INFO: 172.20.0.1:49196 - "POST /node/69d614c74b6493d8616621b2 HTTP/1.0" 202 Accepted 2026-04-08 08:42:42,401 - lava_callback - INFO - Starting async job submit for node 69d614c74b6493d8616621b2 2026-04-08 08:42:42,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:42,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c74b6493d8616621b2 HTTP/1.1" 200 None 2026-04-08 08:42:42,436 - lava_callback - DEBUG - Processing node 69d614c74b6493d8616621b2: baseline-arm-pengutronix 2026-04-08 08:42:42,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:42,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:42,722 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c74b6493d8616621b2/log.txt.gz 2026-04-08 08:42:42,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:42,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:42,982 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c74b6493d8616621b2/lava-logs.yaml 2026-04-08 08:42:42,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:43,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:43,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c74b6493d8616621b2/lava_callback.json.gz 2026-04-08 08:42:43,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:43,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c74b6493d8616621b2 HTTP/1.1" 200 None 2026-04-08 08:42:43,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:43,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:42:43,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:43,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614c74b6493d8616621b2 HTTP/1.1" 200 None 2026-04-08 08:42:43,195 - lava_callback - INFO - Completed processing callback for node 69d614c74b6493d8616621b2 2026-04-08 08:42:43,592 - lava_callback - INFO - Received callback for node 69d614c84b6493d8616621b5 INFO: 172.20.0.1:49198 - "POST /node/69d614c84b6493d8616621b5 HTTP/1.0" 202 Accepted 2026-04-08 08:42:43,598 - lava_callback - INFO - Starting async job submit for node 69d614c84b6493d8616621b5 2026-04-08 08:42:43,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:43,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c84b6493d8616621b5 HTTP/1.1" 200 None 2026-04-08 08:42:43,634 - lava_callback - DEBUG - Processing node 69d614c84b6493d8616621b5: baseline-arm-pengutronix 2026-04-08 08:42:43,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:43,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:43,901 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c84b6493d8616621b5/log.txt.gz 2026-04-08 08:42:43,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:44,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:44,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c84b6493d8616621b5/lava-logs.yaml 2026-04-08 08:42:44,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:44,291 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:44,293 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d614c84b6493d8616621b5/lava_callback.json.gz 2026-04-08 08:42:44,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:44,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c84b6493d8616621b5 HTTP/1.1" 200 None 2026-04-08 08:42:44,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:44,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:42:44,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:44,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614c84b6493d8616621b5 HTTP/1.1" 200 None 2026-04-08 08:42:44,369 - lava_callback - INFO - Completed processing callback for node 69d614c84b6493d8616621b5 INFO: 172.20.0.1:49202 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:42:48,664 - lava_callback - INFO - Received callback for node 69d614a64b6493d86166210f INFO: 172.20.0.1:49218 - "POST /node/69d614a64b6493d86166210f HTTP/1.0" 202 Accepted 2026-04-08 08:42:48,672 - lava_callback - INFO - Starting async job submit for node 69d614a64b6493d86166210f 2026-04-08 08:42:48,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:48,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a64b6493d86166210f HTTP/1.1" 200 None 2026-04-08 08:42:48,719 - lava_callback - DEBUG - Processing node 69d614a64b6493d86166210f: baseline-arm 2026-04-08 08:42:48,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:49,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:49,082 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a64b6493d86166210f/log.txt.gz 2026-04-08 08:42:49,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:49,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:49,397 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a64b6493d86166210f/lava-logs.yaml 2026-04-08 08:42:49,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:42:49,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:42:49,525 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a64b6493d86166210f/lava_callback.json.gz 2026-04-08 08:42:49,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:49,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a64b6493d86166210f HTTP/1.1" 200 None 2026-04-08 08:42:49,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:49,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:42:49,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:49,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614a64b6493d86166210f HTTP/1.1" 200 None 2026-04-08 08:42:49,603 - lava_callback - INFO - Completed processing callback for node 69d614a64b6493d86166210f 2026-04-08 08:42:50,255 - lava_callback - INFO - Received callback for node 69d371f92214054febe380f9 INFO: 172.20.0.1:37988 - "POST /node/69d371f92214054febe380f9 HTTP/1.0" 202 Accepted 2026-04-08 08:42:50,260 - lava_callback - INFO - Starting async job submit for node 69d371f92214054febe380f9 2026-04-08 08:42:50,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:50,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371f92214054febe380f9 HTTP/1.1" 200 None 2026-04-08 08:42:50,276 - lava_callback - DEBUG - Processing node 69d371f92214054febe380f9: baseline-arm-clabbe 2026-04-08 08:42:50,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d371f92214054febe380f9 2026-04-08 08:42:50,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:50,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d371f92214054febe380f9 HTTP/1.1" 200 None 2026-04-08 08:42:50,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:50,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:42:50,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:42:50,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d371f92214054febe380f9 HTTP/1.1" 200 None 2026-04-08 08:42:50,330 - lava_callback - ERROR - Error processing callback for node 69d371f92214054febe380f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37998 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:43:04,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:04,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:43:25,309 - lava_callback - INFO - Received callback for node 69d614c24b6493d86166219c INFO: 172.20.0.1:52806 - "POST /node/69d614c24b6493d86166219c HTTP/1.0" 202 Accepted 2026-04-08 08:43:25,315 - lava_callback - INFO - Starting async job submit for node 69d614c24b6493d86166219c 2026-04-08 08:43:25,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:25,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c24b6493d86166219c HTTP/1.1" 200 None 2026-04-08 08:43:25,355 - lava_callback - DEBUG - Processing node 69d614c24b6493d86166219c: baseline-arm-kontron 2026-04-08 08:43:25,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:43:25,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:43:25,757 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d614c24b6493d86166219c/log.txt.gz 2026-04-08 08:43:25,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:43:26,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:43:26,075 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d614c24b6493d86166219c/lava-logs.yaml 2026-04-08 08:43:26,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:43:26,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:43:26,220 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d614c24b6493d86166219c/lava_callback.json.gz 2026-04-08 08:43:26,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:26,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c24b6493d86166219c HTTP/1.1" 200 None 2026-04-08 08:43:26,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:26,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:43:26,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:26,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614c24b6493d86166219c HTTP/1.1" 200 None 2026-04-08 08:43:26,298 - lava_callback - INFO - Completed processing callback for node 69d614c24b6493d86166219c INFO: 172.20.0.1:52818 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:43:26,526 - lava_callback - INFO - Received callback for node 69d614c44b6493d86166219f INFO: 172.20.0.1:52822 - "POST /node/69d614c44b6493d86166219f HTTP/1.0" 202 Accepted 2026-04-08 08:43:26,532 - lava_callback - INFO - Starting async job submit for node 69d614c44b6493d86166219f 2026-04-08 08:43:26,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:26,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c44b6493d86166219f HTTP/1.1" 200 None 2026-04-08 08:43:26,581 - lava_callback - DEBUG - Processing node 69d614c44b6493d86166219f: baseline-arm-kontron 2026-04-08 08:43:26,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:43:26,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:43:26,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d614c44b6493d86166219f/log.txt.gz 2026-04-08 08:43:26,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:43:27,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:43:27,183 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d614c44b6493d86166219f/lava-logs.yaml 2026-04-08 08:43:27,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:43:27,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:43:27,345 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d614c44b6493d86166219f/lava_callback.json.gz 2026-04-08 08:43:27,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:27,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614c44b6493d86166219f HTTP/1.1" 200 None 2026-04-08 08:43:27,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:27,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:43:27,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:27,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614c44b6493d86166219f HTTP/1.1" 200 None 2026-04-08 08:43:27,431 - lava_callback - INFO - Completed processing callback for node 69d614c44b6493d86166219f 2026-04-08 08:43:30,332 - lava_callback - INFO - Received callback for node 69d3721c2214054febe3810f INFO: 172.20.0.1:53926 - "POST /node/69d3721c2214054febe3810f HTTP/1.0" 202 Accepted 2026-04-08 08:43:30,338 - lava_callback - INFO - Starting async job submit for node 69d3721c2214054febe3810f 2026-04-08 08:43:30,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:30,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3721c2214054febe3810f HTTP/1.1" 200 None 2026-04-08 08:43:30,354 - lava_callback - DEBUG - Processing node 69d3721c2214054febe3810f: baseline-arm-clabbe 2026-04-08 08:43:30,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3721c2214054febe3810f 2026-04-08 08:43:30,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:30,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3721c2214054febe3810f HTTP/1.1" 200 None 2026-04-08 08:43:30,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:30,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:43:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:30,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3721c2214054febe3810f HTTP/1.1" 200 None 2026-04-08 08:43:30,413 - lava_callback - ERROR - Error processing callback for node 69d3721c2214054febe3810f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:43:34,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:34,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56534 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:43:50,298 - lava_callback - INFO - Received callback for node 69d3723e2214054febe3812e INFO: 172.20.0.1:42630 - "POST /node/69d3723e2214054febe3812e HTTP/1.0" 202 Accepted 2026-04-08 08:43:50,307 - lava_callback - INFO - Starting async job submit for node 69d3723e2214054febe3812e 2026-04-08 08:43:50,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:50,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3723e2214054febe3812e HTTP/1.1" 200 None 2026-04-08 08:43:50,346 - lava_callback - DEBUG - Processing node 69d3723e2214054febe3812e: baseline-arm-clabbe 2026-04-08 08:43:50,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3723e2214054febe3812e 2026-04-08 08:43:50,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:50,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3723e2214054febe3812e HTTP/1.1" 200 None 2026-04-08 08:43:50,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:50,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:43:50,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:43:50,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3723e2214054febe3812e HTTP/1.1" 200 None 2026-04-08 08:43:50,448 - lava_callback - ERROR - Error processing callback for node 69d3723e2214054febe3812e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43146 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:44:14,192 - lava_callback - INFO - Received callback for node 69d614dc4b6493d8616621dc INFO: 172.20.0.1:45190 - "POST /node/69d614dc4b6493d8616621dc HTTP/1.0" 202 Accepted 2026-04-08 08:44:14,202 - lava_callback - INFO - Starting async job submit for node 69d614dc4b6493d8616621dc 2026-04-08 08:44:14,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:14,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614dc4b6493d8616621dc HTTP/1.1" 200 None 2026-04-08 08:44:14,287 - lava_callback - DEBUG - Processing node 69d614dc4b6493d8616621dc: ltp-dio 2026-04-08 08:44:14,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45196 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 08:44:15,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:44:15,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69d614dc4b6493d8616621dc/log.txt.gz 2026-04-08 08:44:15,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:44:16,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:44:16,482 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69d614dc4b6493d8616621dc/lava-logs.yaml 2026-04-08 08:44:16,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:44:17,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:44:17,007 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69d614dc4b6493d8616621dc/lava_callback.json.gz 2026-04-08 08:44:17,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:17,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614dc4b6493d8616621dc HTTP/1.1" 200 None 2026-04-08 08:44:17,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:17,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:44:17,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:17,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614dc4b6493d8616621dc HTTP/1.1" 200 None 2026-04-08 08:44:17,144 - lava_callback - INFO - Completed processing callback for node 69d614dc4b6493d8616621dc INFO: 172.20.0.1:48118 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:44:28,607 - lava_callback - INFO - Received callback for node 69d615604b6493d8616622e8 INFO: 172.20.0.1:48134 - "POST /node/69d615604b6493d8616622e8 HTTP/1.0" 202 Accepted 2026-04-08 08:44:28,616 - lava_callback - INFO - Starting async job submit for node 69d615604b6493d8616622e8 2026-04-08 08:44:28,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:28,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615604b6493d8616622e8 HTTP/1.1" 200 None 2026-04-08 08:44:28,650 - lava_callback - DEBUG - Processing node 69d615604b6493d8616622e8: baseline-arm 2026-04-08 08:44:28,660 - lava_callback - WARNING - Failed to upload log 2026-04-08 08:44:28,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:44:29,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:44:29,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d615604b6493d8616622e8/lava-logs.yaml 2026-04-08 08:44:29,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:44:29,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:44:29,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d615604b6493d8616622e8/lava_callback.json.gz 2026-04-08 08:44:29,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:29,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615604b6493d8616622e8 HTTP/1.1" 200 None 2026-04-08 08:44:29,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:29,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:44:29,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:29,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615604b6493d8616622e8 HTTP/1.1" 200 None 2026-04-08 08:44:29,531 - lava_callback - INFO - Completed processing callback for node 69d615604b6493d8616622e8 2026-04-08 08:44:30,366 - lava_callback - INFO - Received callback for node 69d372642214054febe38187 INFO: 172.20.0.1:42740 - "POST /node/69d372642214054febe38187 HTTP/1.0" 202 Accepted 2026-04-08 08:44:30,375 - lava_callback - INFO - Starting async job submit for node 69d372642214054febe38187 2026-04-08 08:44:30,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:30,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372642214054febe38187 HTTP/1.1" 200 None 2026-04-08 08:44:30,404 - lava_callback - DEBUG - Processing node 69d372642214054febe38187: baseline-arm-clabbe 2026-04-08 08:44:30,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372642214054febe38187 2026-04-08 08:44:30,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:30,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372642214054febe38187 HTTP/1.1" 200 None 2026-04-08 08:44:30,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:30,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c903691bac272b322d89 HTTP/1.1" 200 None 2026-04-08 08:44:30,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:30,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372642214054febe38187 HTTP/1.1" 200 None 2026-04-08 08:44:30,509 - lava_callback - ERROR - Error processing callback for node 69d372642214054febe38187: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:31,832 - lava_callback - INFO - Received callback for node 69d372592214054febe38179 INFO: 172.20.0.1:42744 - "POST /node/69d372592214054febe38179 HTTP/1.0" 202 Accepted 2026-04-08 08:44:31,843 - lava_callback - INFO - Starting async job submit for node 69d372592214054febe38179 2026-04-08 08:44:31,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:31,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372592214054febe38179 HTTP/1.1" 200 None 2026-04-08 08:44:31,883 - lava_callback - DEBUG - Processing node 69d372592214054febe38179: baseline-arm-baylibre 2026-04-08 08:44:31,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372592214054febe38179 2026-04-08 08:44:31,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:31,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372592214054febe38179 HTTP/1.1" 200 None 2026-04-08 08:44:31,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:31,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:31,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:31,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372592214054febe38179 HTTP/1.1" 200 None 2026-04-08 08:44:31,980 - lava_callback - ERROR - Error processing callback for node 69d372592214054febe38179: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:32,510 - lava_callback - INFO - Received callback for node 69d3725c2214054febe3817c INFO: 172.20.0.1:42756 - "POST /node/69d3725c2214054febe3817c HTTP/1.0" 202 Accepted 2026-04-08 08:44:32,519 - lava_callback - INFO - Starting async job submit for node 69d3725c2214054febe3817c 2026-04-08 08:44:32,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:32,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3725c2214054febe3817c HTTP/1.1" 200 None 2026-04-08 08:44:32,549 - lava_callback - DEBUG - Processing node 69d3725c2214054febe3817c: baseline-arm-baylibre 2026-04-08 08:44:32,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3725c2214054febe3817c 2026-04-08 08:44:32,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:32,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3725c2214054febe3817c HTTP/1.1" 200 None 2026-04-08 08:44:32,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:32,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:32,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:32,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3725c2214054febe3817c HTTP/1.1" 200 None 2026-04-08 08:44:32,657 - lava_callback - ERROR - Error processing callback for node 69d3725c2214054febe3817c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:33,212 - lava_callback - INFO - Received callback for node 69d3725e2214054febe3817f INFO: 172.20.0.1:42770 - "POST /node/69d3725e2214054febe3817f HTTP/1.0" 202 Accepted 2026-04-08 08:44:33,221 - lava_callback - INFO - Starting async job submit for node 69d3725e2214054febe3817f 2026-04-08 08:44:33,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:33,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3725e2214054febe3817f HTTP/1.1" 200 None 2026-04-08 08:44:33,254 - lava_callback - DEBUG - Processing node 69d3725e2214054febe3817f: baseline-arm-baylibre 2026-04-08 08:44:33,254 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3725e2214054febe3817f 2026-04-08 08:44:33,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:33,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3725e2214054febe3817f HTTP/1.1" 200 None 2026-04-08 08:44:33,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:33,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:33,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:33,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3725e2214054febe3817f HTTP/1.1" 200 None 2026-04-08 08:44:33,344 - lava_callback - ERROR - Error processing callback for node 69d3725e2214054febe3817f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:33,880 - lava_callback - INFO - Received callback for node 69d372612214054febe38182 INFO: 172.20.0.1:42776 - "POST /node/69d372612214054febe38182 HTTP/1.0" 202 Accepted 2026-04-08 08:44:33,889 - lava_callback - INFO - Starting async job submit for node 69d372612214054febe38182 2026-04-08 08:44:33,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:33,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372612214054febe38182 HTTP/1.1" 200 None 2026-04-08 08:44:33,917 - lava_callback - DEBUG - Processing node 69d372612214054febe38182: baseline-arm-baylibre 2026-04-08 08:44:33,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372612214054febe38182 2026-04-08 08:44:33,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:33,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372612214054febe38182 HTTP/1.1" 200 None 2026-04-08 08:44:33,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:34,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:34,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:34,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372612214054febe38182 HTTP/1.1" 200 None 2026-04-08 08:44:34,055 - lava_callback - ERROR - Error processing callback for node 69d372612214054febe38182: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:34,532 - lava_callback - INFO - Received callback for node 69d372672214054febe38192 INFO: 172.20.0.1:42786 - "POST /node/69d372672214054febe38192 HTTP/1.0" 202 Accepted 2026-04-08 08:44:34,541 - lava_callback - INFO - Starting async job submit for node 69d372672214054febe38192 2026-04-08 08:44:34,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:34,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372672214054febe38192 HTTP/1.1" 200 None 2026-04-08 08:44:34,571 - lava_callback - DEBUG - Processing node 69d372672214054febe38192: baseline-arm-baylibre 2026-04-08 08:44:34,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372672214054febe38192 2026-04-08 08:44:34,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:34,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372672214054febe38192 HTTP/1.1" 200 None 2026-04-08 08:44:34,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:34,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:34,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:34,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372672214054febe38192 HTTP/1.1" 200 None 2026-04-08 08:44:34,680 - lava_callback - ERROR - Error processing callback for node 69d372672214054febe38192: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:34,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:34,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-08 08:44:35,206 - lava_callback - INFO - Received callback for node 69d3726a2214054febe381ae INFO: 172.20.0.1:42796 - "POST /node/69d3726a2214054febe381ae HTTP/1.0" 202 Accepted 2026-04-08 08:44:35,214 - lava_callback - INFO - Starting async job submit for node 69d3726a2214054febe381ae 2026-04-08 08:44:35,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:35,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3726a2214054febe381ae HTTP/1.1" 200 None 2026-04-08 08:44:35,256 - lava_callback - DEBUG - Processing node 69d3726a2214054febe381ae: baseline-arm-baylibre 2026-04-08 08:44:35,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3726a2214054febe381ae 2026-04-08 08:44:35,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:35,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3726a2214054febe381ae HTTP/1.1" 200 None 2026-04-08 08:44:35,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:35,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:35,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:35,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3726a2214054febe381ae HTTP/1.1" 200 None 2026-04-08 08:44:35,347 - lava_callback - ERROR - Error processing callback for node 69d3726a2214054febe381ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:44:50,342 - lava_callback - INFO - Received callback for node 69d372792214054febe381c3 INFO: 172.20.0.1:38720 - "POST /node/69d372792214054febe381c3 HTTP/1.0" 202 Accepted 2026-04-08 08:44:50,347 - lava_callback - INFO - Starting async job submit for node 69d372792214054febe381c3 2026-04-08 08:44:50,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:50,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372792214054febe381c3 HTTP/1.1" 200 None 2026-04-08 08:44:50,371 - lava_callback - DEBUG - Processing node 69d372792214054febe381c3: baseline-arm-clabbe 2026-04-08 08:44:50,371 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372792214054febe381c3 2026-04-08 08:44:50,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:50,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372792214054febe381c3 HTTP/1.1" 200 None 2026-04-08 08:44:50,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:50,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:50,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:50,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372792214054febe381c3 HTTP/1.1" 200 None 2026-04-08 08:44:50,438 - lava_callback - ERROR - Error processing callback for node 69d372792214054febe381c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:51,027 - lava_callback - INFO - Received callback for node 69d3727d2214054febe381ca INFO: 172.20.0.1:38726 - "POST /node/69d3727d2214054febe381ca HTTP/1.0" 202 Accepted 2026-04-08 08:44:51,033 - lava_callback - INFO - Starting async job submit for node 69d3727d2214054febe381ca 2026-04-08 08:44:51,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3727d2214054febe381ca HTTP/1.1" 200 None 2026-04-08 08:44:51,058 - lava_callback - DEBUG - Processing node 69d3727d2214054febe381ca: baseline-arm-clabbe 2026-04-08 08:44:51,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3727d2214054febe381ca 2026-04-08 08:44:51,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3727d2214054febe381ca HTTP/1.1" 200 None 2026-04-08 08:44:51,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:51,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3727d2214054febe381ca HTTP/1.1" 200 None 2026-04-08 08:44:51,115 - lava_callback - ERROR - Error processing callback for node 69d3727d2214054febe381ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:44:51,716 - lava_callback - INFO - Received callback for node 69d3727f2214054febe381ce INFO: 172.20.0.1:38736 - "POST /node/69d3727f2214054febe381ce HTTP/1.0" 202 Accepted 2026-04-08 08:44:51,721 - lava_callback - INFO - Starting async job submit for node 69d3727f2214054febe381ce 2026-04-08 08:44:51,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3727f2214054febe381ce HTTP/1.1" 200 None 2026-04-08 08:44:51,747 - lava_callback - DEBUG - Processing node 69d3727f2214054febe381ce: baseline-arm-clabbe 2026-04-08 08:44:51,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3727f2214054febe381ce 2026-04-08 08:44:51,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3727f2214054febe381ce HTTP/1.1" 200 None 2026-04-08 08:44:51,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:44:51,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:44:51,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3727f2214054febe381ce HTTP/1.1" 200 None 2026-04-08 08:44:51,858 - lava_callback - ERROR - Error processing callback for node 69d3727f2214054febe381ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34568 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:45:10,336 - lava_callback - INFO - Received callback for node 69d372802214054febe381d5 INFO: 172.20.0.1:56844 - "POST /node/69d372802214054febe381d5 HTTP/1.0" 202 Accepted 2026-04-08 08:45:10,345 - lava_callback - INFO - Starting async job submit for node 69d372802214054febe381d5 2026-04-08 08:45:10,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:10,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372802214054febe381d5 HTTP/1.1" 200 None 2026-04-08 08:45:10,378 - lava_callback - DEBUG - Processing node 69d372802214054febe381d5: baseline-arm-clabbe 2026-04-08 08:45:10,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372802214054febe381d5 2026-04-08 08:45:10,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:10,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372802214054febe381d5 HTTP/1.1" 200 None 2026-04-08 08:45:10,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:10,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:45:10,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:10,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372802214054febe381d5 HTTP/1.1" 200 None 2026-04-08 08:45:10,481 - lava_callback - ERROR - Error processing callback for node 69d372802214054febe381d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:45:11,029 - lava_callback - INFO - Received callback for node 69d372822214054febe381d8 INFO: 172.20.0.1:56852 - "POST /node/69d372822214054febe381d8 HTTP/1.0" 202 Accepted 2026-04-08 08:45:11,038 - lava_callback - INFO - Starting async job submit for node 69d372822214054febe381d8 2026-04-08 08:45:11,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372822214054febe381d8 HTTP/1.1" 200 None 2026-04-08 08:45:11,094 - lava_callback - DEBUG - Processing node 69d372822214054febe381d8: baseline-arm-clabbe 2026-04-08 08:45:11,094 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372822214054febe381d8 2026-04-08 08:45:11,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372822214054febe381d8 HTTP/1.1" 200 None 2026-04-08 08:45:11,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:45:11,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372822214054febe381d8 HTTP/1.1" 200 None 2026-04-08 08:45:11,223 - lava_callback - ERROR - Error processing callback for node 69d372822214054febe381d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:45:11,678 - lava_callback - INFO - Received callback for node 69d372852214054febe381dd INFO: 172.20.0.1:56866 - "POST /node/69d372852214054febe381dd HTTP/1.0" 202 Accepted 2026-04-08 08:45:11,686 - lava_callback - INFO - Starting async job submit for node 69d372852214054febe381dd 2026-04-08 08:45:11,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,708 - lava_callback - INFO - Received callback for node 69d372842214054febe381db INFO: 172.20.0.1:56880 - "POST /node/69d372842214054febe381db HTTP/1.0" 202 Accepted 2026-04-08 08:45:11,720 - lava_callback - INFO - Starting async job submit for node 69d372842214054febe381db 2026-04-08 08:45:11,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372852214054febe381dd HTTP/1.1" 200 None 2026-04-08 08:45:11,730 - lava_callback - DEBUG - Processing node 69d372852214054febe381dd: baseline-arm-baylibre 2026-04-08 08:45:11,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372852214054febe381dd 2026-04-08 08:45:11,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372842214054febe381db HTTP/1.1" 200 None 2026-04-08 08:45:11,752 - lava_callback - DEBUG - Processing node 69d372842214054febe381db: baseline-arm-clabbe 2026-04-08 08:45:11,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372842214054febe381db 2026-04-08 08:45:11,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372852214054febe381dd HTTP/1.1" 200 None 2026-04-08 08:45:11,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 08:45:11,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372842214054febe381db HTTP/1.1" 200 None 2026-04-08 08:45:11,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:45:11,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:11,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372852214054febe381dd HTTP/1.1" 200 None 2026-04-08 08:45:11,854 - lava_callback - ERROR - Error processing callback for node 69d372852214054febe381dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:45:11,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372842214054febe381db HTTP/1.1" 200 None 2026-04-08 08:45:11,922 - lava_callback - ERROR - Error processing callback for node 69d372842214054febe381db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:45:12,405 - lava_callback - INFO - Received callback for node 69d372852214054febe381e0 INFO: 172.20.0.1:56894 - "POST /node/69d372852214054febe381e0 HTTP/1.0" 202 Accepted 2026-04-08 08:45:12,414 - lava_callback - INFO - Starting async job submit for node 69d372852214054febe381e0 2026-04-08 08:45:12,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:12,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372852214054febe381e0 HTTP/1.1" 200 None 2026-04-08 08:45:12,446 - lava_callback - DEBUG - Processing node 69d372852214054febe381e0: baseline-arm-clabbe 2026-04-08 08:45:12,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372852214054febe381e0 2026-04-08 08:45:12,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:12,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372852214054febe381e0 HTTP/1.1" 200 None 2026-04-08 08:45:12,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:12,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:45:12,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:12,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372852214054febe381e0 HTTP/1.1" 200 None 2026-04-08 08:45:12,538 - lava_callback - ERROR - Error processing callback for node 69d372852214054febe381e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:45:13,087 - lava_callback - INFO - Received callback for node 69d372872214054febe381ff INFO: 172.20.0.1:56900 - "POST /node/69d372872214054febe381ff HTTP/1.0" 202 Accepted 2026-04-08 08:45:13,096 - lava_callback - INFO - Starting async job submit for node 69d372872214054febe381ff 2026-04-08 08:45:13,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372872214054febe381ff HTTP/1.1" 200 None 2026-04-08 08:45:13,126 - lava_callback - DEBUG - Processing node 69d372872214054febe381ff: baseline-arm-clabbe 2026-04-08 08:45:13,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372872214054febe381ff 2026-04-08 08:45:13,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372872214054febe381ff HTTP/1.1" 200 None 2026-04-08 08:45:13,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:45:13,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372872214054febe381ff HTTP/1.1" 200 None 2026-04-08 08:45:13,219 - lava_callback - ERROR - Error processing callback for node 69d372872214054febe381ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:45:13,827 - lava_callback - INFO - Received callback for node 69d372892214054febe38203 INFO: 172.20.0.1:56916 - "POST /node/69d372892214054febe38203 HTTP/1.0" 202 Accepted 2026-04-08 08:45:13,837 - lava_callback - INFO - Starting async job submit for node 69d372892214054febe38203 2026-04-08 08:45:13,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372892214054febe38203 HTTP/1.1" 200 None 2026-04-08 08:45:13,873 - lava_callback - DEBUG - Processing node 69d372892214054febe38203: baseline-arm-clabbe 2026-04-08 08:45:13,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372892214054febe38203 2026-04-08 08:45:13,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372892214054febe38203 HTTP/1.1" 200 None 2026-04-08 08:45:13,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c222214054febe37d62 HTTP/1.1" 200 None 2026-04-08 08:45:13,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:13,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372892214054febe38203 HTTP/1.1" 200 None 2026-04-08 08:45:13,971 - lava_callback - ERROR - Error processing callback for node 69d372892214054febe38203: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:45:31,686 - lava_callback - INFO - Received callback for node 69d372a42214054febe3826b INFO: 172.20.0.1:53456 - "POST /node/69d372a42214054febe3826b HTTP/1.0" 202 Accepted 2026-04-08 08:45:31,695 - lava_callback - INFO - Starting async job submit for node 69d372a42214054febe3826b 2026-04-08 08:45:31,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:31,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372a42214054febe3826b HTTP/1.1" 200 None 2026-04-08 08:45:31,727 - lava_callback - DEBUG - Processing node 69d372a42214054febe3826b: baseline-arm-baylibre 2026-04-08 08:45:31,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d372a42214054febe3826b 2026-04-08 08:45:31,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:31,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d372a42214054febe3826b HTTP/1.1" 200 None 2026-04-08 08:45:31,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:31,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 08:45:31,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:31,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d372a42214054febe3826b HTTP/1.1" 200 None 2026-04-08 08:45:31,826 - lava_callback - ERROR - Error processing callback for node 69d372a42214054febe3826b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:45:52,958 - lava_callback - INFO - Received callback for node 69d614e04b6493d861662207 INFO: 172.20.0.1:51598 - "POST /node/69d614e04b6493d861662207 HTTP/1.0" 202 Accepted 2026-04-08 08:45:52,969 - lava_callback - INFO - Starting async job submit for node 69d614e04b6493d861662207 2026-04-08 08:45:53,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:53,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e04b6493d861662207 HTTP/1.1" 200 None 2026-04-08 08:45:53,069 - lava_callback - DEBUG - Processing node 69d614e04b6493d861662207: ltp-input 2026-04-08 08:45:53,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:45:53,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:45:53,979 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69d614e04b6493d861662207/log.txt.gz 2026-04-08 08:45:54,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:45:54,595 - lava_callback - INFO - Received callback for node 69d614de4b6493d861662204 INFO: 172.20.0.1:51614 - "POST /node/69d614de4b6493d861662204 HTTP/1.0" 202 Accepted 2026-04-08 08:45:54,605 - lava_callback - INFO - Starting async job submit for node 69d614de4b6493d861662204 2026-04-08 08:45:54,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:54,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614de4b6493d861662204 HTTP/1.1" 200 None 2026-04-08 08:45:54,695 - lava_callback - DEBUG - Processing node 69d614de4b6493d861662204: ltp-fsx 2026-04-08 08:45:54,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:45:54,867 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69d614e04b6493d861662207/lava-logs.yaml 2026-04-08 08:45:54,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:45:55,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:45:55,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69d614e04b6493d861662207/lava_callback.json.gz 2026-04-08 08:45:55,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:55,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e04b6493d861662207 HTTP/1.1" 200 None 2026-04-08 08:45:55,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:55,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:45:55,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:55,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:45:55,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614e04b6493d861662207 HTTP/1.1" 200 None 2026-04-08 08:45:55,545 - lava_callback - INFO - Completed processing callback for node 69d614e04b6493d861662207 2026-04-08 08:45:56,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:45:56,023 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69d614de4b6493d861662204/log.txt.gz 2026-04-08 08:45:56,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51620 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:45:56,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:45:56,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69d614de4b6493d861662204/lava-logs.yaml 2026-04-08 08:45:56,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:45:56,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:45:56,941 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69d614de4b6493d861662204/lava_callback.json.gz 2026-04-08 08:45:56,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:56,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614de4b6493d861662204 HTTP/1.1" 200 None 2026-04-08 08:45:56,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:57,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:45:57,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:45:57,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614de4b6493d861662204 HTTP/1.1" 200 None 2026-04-08 08:45:57,062 - lava_callback - INFO - Completed processing callback for node 69d614de4b6493d861662204 2026-04-08 08:46:04,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:04,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:46:13,204 - lava_callback - INFO - Received callback for node 69d614e14b6493d86166220a INFO: 172.20.0.1:57832 - "POST /node/69d614e14b6493d86166220a HTTP/1.0" 202 Accepted 2026-04-08 08:46:13,223 - lava_callback - INFO - Starting async job submit for node 69d614e14b6493d86166220a 2026-04-08 08:46:13,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:13,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e14b6493d86166220a HTTP/1.1" 200 None 2026-04-08 08:46:13,362 - lava_callback - DEBUG - Processing node 69d614e14b6493d86166220a: ltp-ipc 2026-04-08 08:46:13,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:14,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:14,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69d614e14b6493d86166220a/log.txt.gz 2026-04-08 08:46:14,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57838 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 08:46:15,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:15,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69d614e14b6493d86166220a/lava-logs.yaml 2026-04-08 08:46:15,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:15,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:15,546 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69d614e14b6493d86166220a/lava_callback.json.gz 2026-04-08 08:46:15,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:15,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e14b6493d86166220a HTTP/1.1" 200 None 2026-04-08 08:46:15,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:15,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:46:15,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:15,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614e14b6493d86166220a HTTP/1.1" 200 None 2026-04-08 08:46:15,660 - lava_callback - INFO - Completed processing callback for node 69d614e14b6493d86166220a INFO: 172.20.0.1:57844 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:46:20,511 - lava_callback - INFO - Received callback for node 69d615c14b6493d86166237f INFO: 172.20.0.1:54560 - "POST /node/69d615c14b6493d86166237f HTTP/1.0" 202 Accepted 2026-04-08 08:46:20,521 - lava_callback - INFO - Starting async job submit for node 69d615c14b6493d86166237f 2026-04-08 08:46:20,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:20,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c14b6493d86166237f HTTP/1.1" 200 None 2026-04-08 08:46:20,584 - lava_callback - DEBUG - Processing node 69d615c14b6493d86166237f: baseline-arm64 2026-04-08 08:46:20,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:21,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:21,050 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c14b6493d86166237f/log.txt.gz 2026-04-08 08:46:21,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:21,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:21,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c14b6493d86166237f/lava-logs.yaml 2026-04-08 08:46:21,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:21,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:21,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c14b6493d86166237f/lava_callback.json.gz 2026-04-08 08:46:21,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:21,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c14b6493d86166237f HTTP/1.1" 200 None 2026-04-08 08:46:21,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:21,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:46:21,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:22,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c14b6493d86166237f HTTP/1.1" 200 None 2026-04-08 08:46:22,010 - lava_callback - INFO - Completed processing callback for node 69d615c14b6493d86166237f INFO: 172.20.0.1:54570 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:46:34,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:34,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54428 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:46:44,969 - lava_callback - INFO - Received callback for node 69d615cf4b6493d8616623df INFO: 172.20.0.1:39434 - "POST /node/69d615cf4b6493d8616623df HTTP/1.0" 202 Accepted 2026-04-08 08:46:44,980 - lava_callback - INFO - Starting async job submit for node 69d615cf4b6493d8616623df 2026-04-08 08:46:45,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:45,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615cf4b6493d8616623df HTTP/1.1" 200 None 2026-04-08 08:46:45,023 - lava_callback - DEBUG - Processing node 69d615cf4b6493d8616623df: baseline-arm64 2026-04-08 08:46:45,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:45,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:45,354 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cf4b6493d8616623df/log.txt.gz 2026-04-08 08:46:45,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:45,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:45,985 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cf4b6493d8616623df/lava-logs.yaml 2026-04-08 08:46:45,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39436 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:46:46,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:46,733 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cf4b6493d8616623df/lava_callback.json.gz 2026-04-08 08:46:46,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:46,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615cf4b6493d8616623df HTTP/1.1" 200 None 2026-04-08 08:46:46,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:46,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:46:46,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:46,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615cf4b6493d8616623df HTTP/1.1" 200 None 2026-04-08 08:46:46,822 - lava_callback - INFO - Completed processing callback for node 69d615cf4b6493d8616623df 2026-04-08 08:46:49,920 - lava_callback - INFO - Received callback for node 69d615e24b6493d861662413 INFO: 172.20.0.1:55074 - "POST /node/69d615e24b6493d861662413 HTTP/1.0" 202 Accepted 2026-04-08 08:46:49,927 - lava_callback - INFO - Starting async job submit for node 69d615e24b6493d861662413 2026-04-08 08:46:49,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:49,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e24b6493d861662413 HTTP/1.1" 200 None 2026-04-08 08:46:49,945 - lava_callback - DEBUG - Processing node 69d615e24b6493d861662413: baseline-nfs-arm64 2026-04-08 08:46:49,947 - lava_callback - WARNING - Failed to upload log 2026-04-08 08:46:49,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:50,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:50,062 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e24b6493d861662413/lava-logs.yaml 2026-04-08 08:46:50,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:46:50,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:46:50,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e24b6493d861662413/lava_callback.json.gz 2026-04-08 08:46:50,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:50,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e24b6493d861662413 HTTP/1.1" 200 None 2026-04-08 08:46:50,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:50,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:46:50,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:50,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615e24b6493d861662413 HTTP/1.1" 200 None 2026-04-08 08:46:50,534 - lava_callback - INFO - Completed processing callback for node 69d615e24b6493d861662413 INFO: 172.20.0.1:55080 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:46:59,667 - lava_callback - INFO - Received callback for node 69d615c64b6493d861662396 INFO: 172.20.0.1:55084 - "POST /node/69d615c64b6493d861662396 HTTP/1.0" 202 Accepted 2026-04-08 08:46:59,678 - lava_callback - INFO - Starting async job submit for node 69d615c64b6493d861662396 2026-04-08 08:46:59,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:46:59,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c64b6493d861662396 HTTP/1.1" 200 None 2026-04-08 08:46:59,760 - lava_callback - DEBUG - Processing node 69d615c64b6493d861662396: baseline-arm64 2026-04-08 08:47:00,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:00,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:00,383 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c64b6493d861662396/log.txt.gz 2026-04-08 08:47:00,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:00,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:00,976 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c64b6493d861662396/lava-logs.yaml 2026-04-08 08:47:00,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:01,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:01,123 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c64b6493d861662396/lava_callback.json.gz 2026-04-08 08:47:01,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:01,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c64b6493d861662396 HTTP/1.1" 200 None 2026-04-08 08:47:01,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:01,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:01,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:01,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c64b6493d861662396 HTTP/1.1" 200 None 2026-04-08 08:47:01,342 - lava_callback - INFO - Completed processing callback for node 69d615c64b6493d861662396 2026-04-08 08:47:04,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:04,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-08 08:47:05,970 - lava_callback - INFO - Received callback for node 69d615c54b6493d861662393 INFO: 172.20.0.1:39086 - "POST /node/69d615c54b6493d861662393 HTTP/1.0" 202 Accepted 2026-04-08 08:47:05,981 - lava_callback - INFO - Starting async job submit for node 69d615c54b6493d861662393 2026-04-08 08:47:06,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:06,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c54b6493d861662393 HTTP/1.1" 200 None 2026-04-08 08:47:06,100 - lava_callback - DEBUG - Processing node 69d615c54b6493d861662393: baseline-arm64 2026-04-08 08:47:06,354 - lava_callback - WARNING - Failed to upload log 2026-04-08 08:47:06,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39090 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:47:06,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:06,629 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c54b6493d861662393/lava-logs.yaml 2026-04-08 08:47:06,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:06,734 - lava_callback - INFO - Received callback for node 69d615c24b6493d861662385 INFO: 172.20.0.1:39098 - "POST /node/69d615c24b6493d861662385 HTTP/1.0" 202 Accepted 2026-04-08 08:47:06,745 - lava_callback - INFO - Starting async job submit for node 69d615c24b6493d861662385 2026-04-08 08:47:06,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:06,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c24b6493d861662385 HTTP/1.1" 200 None 2026-04-08 08:47:06,813 - lava_callback - DEBUG - Processing node 69d615c24b6493d861662385: baseline-arm64 2026-04-08 08:47:07,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:07,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c54b6493d861662393/lava_callback.json.gz 2026-04-08 08:47:07,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:07,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:07,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c54b6493d861662393 HTTP/1.1" 200 None 2026-04-08 08:47:07,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:07,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:07,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:07,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c54b6493d861662393 HTTP/1.1" 200 None 2026-04-08 08:47:07,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:07,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c24b6493d861662385/log.txt.gz 2026-04-08 08:47:07,563 - lava_callback - INFO - Completed processing callback for node 69d615c54b6493d861662393 2026-04-08 08:47:07,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:08,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:08,261 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c24b6493d861662385/lava-logs.yaml 2026-04-08 08:47:08,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:08,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:08,403 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c24b6493d861662385/lava_callback.json.gz 2026-04-08 08:47:08,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:08,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c24b6493d861662385 HTTP/1.1" 200 None 2026-04-08 08:47:08,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:08,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:08,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:08,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c24b6493d861662385 HTTP/1.1" 200 None 2026-04-08 08:47:08,494 - lava_callback - INFO - Completed processing callback for node 69d615c24b6493d861662385 INFO: 172.20.0.1:37830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37840 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:47:20,201 - lava_callback - INFO - Received callback for node 69d615c94b6493d86166239c INFO: 172.20.0.1:57104 - "POST /node/69d615c94b6493d86166239c HTTP/1.0" 202 Accepted 2026-04-08 08:47:20,210 - lava_callback - INFO - Starting async job submit for node 69d615c94b6493d86166239c 2026-04-08 08:47:20,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:20,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c94b6493d86166239c HTTP/1.1" 200 None 2026-04-08 08:47:20,311 - lava_callback - DEBUG - Processing node 69d615c94b6493d86166239c: baseline-arm64 2026-04-08 08:47:20,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:21,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:21,039 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c94b6493d86166239c/log.txt.gz 2026-04-08 08:47:21,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:21,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:21,617 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c94b6493d86166239c/lava-logs.yaml 2026-04-08 08:47:21,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:21,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:21,963 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c94b6493d86166239c/lava_callback.json.gz 2026-04-08 08:47:21,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:22,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c94b6493d86166239c HTTP/1.1" 200 None 2026-04-08 08:47:22,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:22,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:22,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:22,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c94b6493d86166239c HTTP/1.1" 200 None 2026-04-08 08:47:22,073 - lava_callback - INFO - Completed processing callback for node 69d615c94b6493d86166239c 2026-04-08 08:47:26,106 - lava_callback - INFO - Received callback for node 69d615cb4b6493d86166239f INFO: 172.20.0.1:57120 - "POST /node/69d615cb4b6493d86166239f HTTP/1.0" 202 Accepted 2026-04-08 08:47:26,112 - lava_callback - INFO - Starting async job submit for node 69d615cb4b6493d86166239f 2026-04-08 08:47:26,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:26,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615cb4b6493d86166239f HTTP/1.1" 200 None 2026-04-08 08:47:26,158 - lava_callback - DEBUG - Processing node 69d615cb4b6493d86166239f: baseline-arm64 INFO: 172.20.0.1:57124 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:47:26,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:26,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:26,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cb4b6493d86166239f/log.txt.gz 2026-04-08 08:47:26,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:27,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:27,095 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cb4b6493d86166239f/lava-logs.yaml 2026-04-08 08:47:27,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:27,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:27,226 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cb4b6493d86166239f/lava_callback.json.gz 2026-04-08 08:47:27,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:27,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615cb4b6493d86166239f HTTP/1.1" 200 None 2026-04-08 08:47:27,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:27,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:27,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:27,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615cb4b6493d86166239f HTTP/1.1" 200 None 2026-04-08 08:47:27,317 - lava_callback - INFO - Completed processing callback for node 69d615cb4b6493d86166239f 2026-04-08 08:47:30,445 - lava_callback - INFO - Received callback for node 69d615c34b6493d86166238c INFO: 172.20.0.1:34458 - "POST /node/69d615c34b6493d86166238c HTTP/1.0" 202 Accepted 2026-04-08 08:47:30,451 - lava_callback - INFO - Starting async job submit for node 69d615c34b6493d86166238c 2026-04-08 08:47:30,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:30,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c34b6493d86166238c HTTP/1.1" 200 None 2026-04-08 08:47:30,494 - lava_callback - DEBUG - Processing node 69d615c34b6493d86166238c: baseline-arm64 2026-04-08 08:47:30,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:30,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:30,862 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c34b6493d86166238c/log.txt.gz 2026-04-08 08:47:31,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:31,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:31,167 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c34b6493d86166238c/lava-logs.yaml 2026-04-08 08:47:31,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:31,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:31,317 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c34b6493d86166238c/lava_callback.json.gz 2026-04-08 08:47:31,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:31,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c34b6493d86166238c HTTP/1.1" 200 None 2026-04-08 08:47:31,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:31,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:31,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:31,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c34b6493d86166238c HTTP/1.1" 200 None 2026-04-08 08:47:31,396 - lava_callback - INFO - Completed processing callback for node 69d615c34b6493d86166238c 2026-04-08 08:47:31,845 - lava_callback - INFO - Received callback for node 69d373122214054febe3843f INFO: 172.20.0.1:34472 - "POST /node/69d373122214054febe3843f HTTP/1.0" 202 Accepted 2026-04-08 08:47:31,851 - lava_callback - INFO - Starting async job submit for node 69d373122214054febe3843f 2026-04-08 08:47:31,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:31,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373122214054febe3843f HTTP/1.1" 200 None 2026-04-08 08:47:31,869 - lava_callback - DEBUG - Processing node 69d373122214054febe3843f: baseline-arm64-baylibre 2026-04-08 08:47:31,869 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d373122214054febe3843f 2026-04-08 08:47:31,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:31,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373122214054febe3843f HTTP/1.1" 200 None 2026-04-08 08:47:31,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:31,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:31,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:31,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d373122214054febe3843f HTTP/1.1" 200 None 2026-04-08 08:47:31,922 - lava_callback - ERROR - Error processing callback for node 69d373122214054febe3843f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:32,266 - lava_callback - INFO - Received callback for node 69d615d04b6493d8616623e2 INFO: 172.20.0.1:34478 - "POST /node/69d615d04b6493d8616623e2 HTTP/1.0" 202 Accepted 2026-04-08 08:47:32,272 - lava_callback - INFO - Starting async job submit for node 69d615d04b6493d8616623e2 2026-04-08 08:47:32,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:32,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d04b6493d8616623e2 HTTP/1.1" 200 None 2026-04-08 08:47:32,315 - lava_callback - DEBUG - Processing node 69d615d04b6493d8616623e2: baseline-arm64 2026-04-08 08:47:32,525 - lava_callback - INFO - Received callback for node 69d373132214054febe38444 INFO: 172.20.0.1:34494 - "POST /node/69d373132214054febe38444 HTTP/1.0" 202 Accepted 2026-04-08 08:47:32,546 - lava_callback - INFO - Starting async job submit for node 69d373132214054febe38444 2026-04-08 08:47:32,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:32,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:32,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373132214054febe38444 HTTP/1.1" 200 None 2026-04-08 08:47:32,579 - lava_callback - DEBUG - Processing node 69d373132214054febe38444: baseline-arm64-baylibre 2026-04-08 08:47:32,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d373132214054febe38444 2026-04-08 08:47:32,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:32,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373132214054febe38444 HTTP/1.1" 200 None 2026-04-08 08:47:32,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:32,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:32,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:32,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d373132214054febe38444 HTTP/1.1" 200 None 2026-04-08 08:47:32,640 - lava_callback - ERROR - Error processing callback for node 69d373132214054febe38444: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:32,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:32,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d04b6493d8616623e2/log.txt.gz 2026-04-08 08:47:32,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:33,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:33,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d04b6493d8616623e2/lava-logs.yaml 2026-04-08 08:47:33,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:33,208 - lava_callback - INFO - Received callback for node 69d373152214054febe38449 INFO: 172.20.0.1:34496 - "POST /node/69d373152214054febe38449 HTTP/1.0" 202 Accepted 2026-04-08 08:47:33,214 - lava_callback - INFO - Starting async job submit for node 69d373152214054febe38449 2026-04-08 08:47:33,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:33,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373152214054febe38449 HTTP/1.1" 200 None 2026-04-08 08:47:33,235 - lava_callback - DEBUG - Processing node 69d373152214054febe38449: baseline-arm64-baylibre 2026-04-08 08:47:33,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d373152214054febe38449 2026-04-08 08:47:33,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,236 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d04b6493d8616623e2/lava_callback.json.gz 2026-04-08 08:47:33,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373152214054febe38449 HTTP/1.1" 200 None 2026-04-08 08:47:33,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d04b6493d8616623e2 HTTP/1.1" 200 None 2026-04-08 08:47:33,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:33,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:33,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d373152214054febe38449 HTTP/1.1" 200 None 2026-04-08 08:47:33,321 - lava_callback - ERROR - Error processing callback for node 69d373152214054febe38449: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:33,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615d04b6493d8616623e2 HTTP/1.1" 200 None 2026-04-08 08:47:33,353 - lava_callback - INFO - Completed processing callback for node 69d615d04b6493d8616623e2 2026-04-08 08:47:33,875 - lava_callback - INFO - Received callback for node 69d373172214054febe38450 INFO: 172.20.0.1:34508 - "POST /node/69d373172214054febe38450 HTTP/1.0" 202 Accepted 2026-04-08 08:47:33,881 - lava_callback - INFO - Starting async job submit for node 69d373172214054febe38450 2026-04-08 08:47:33,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373172214054febe38450 HTTP/1.1" 200 None 2026-04-08 08:47:33,897 - lava_callback - DEBUG - Processing node 69d373172214054febe38450: baseline-arm64-baylibre 2026-04-08 08:47:33,897 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d373172214054febe38450 2026-04-08 08:47:33,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373172214054febe38450 HTTP/1.1" 200 None 2026-04-08 08:47:33,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:33,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:33,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d373172214054febe38450 HTTP/1.1" 200 None 2026-04-08 08:47:33,951 - lava_callback - ERROR - Error processing callback for node 69d373172214054febe38450: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:34,573 - lava_callback - INFO - Received callback for node 69d373182214054febe38485 INFO: 172.20.0.1:34522 - "POST /node/69d373182214054febe38485 HTTP/1.0" 202 Accepted 2026-04-08 08:47:34,579 - lava_callback - INFO - Starting async job submit for node 69d373182214054febe38485 2026-04-08 08:47:34,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:34,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373182214054febe38485 HTTP/1.1" 200 None 2026-04-08 08:47:34,600 - lava_callback - DEBUG - Processing node 69d373182214054febe38485: baseline-arm64-baylibre 2026-04-08 08:47:34,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d373182214054febe38485 2026-04-08 08:47:34,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:34,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373182214054febe38485 HTTP/1.1" 200 None 2026-04-08 08:47:34,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:34,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:34,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:34,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d373182214054febe38485 HTTP/1.1" 200 None 2026-04-08 08:47:34,676 - lava_callback - ERROR - Error processing callback for node 69d373182214054febe38485: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:34,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:34,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-08 08:47:35,223 - lava_callback - INFO - Received callback for node 69d3731a2214054febe38488 INFO: 172.20.0.1:34534 - "POST /node/69d3731a2214054febe38488 HTTP/1.0" 202 Accepted 2026-04-08 08:47:35,228 - lava_callback - INFO - Starting async job submit for node 69d3731a2214054febe38488 2026-04-08 08:47:35,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:35,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3731a2214054febe38488 HTTP/1.1" 200 None 2026-04-08 08:47:35,244 - lava_callback - DEBUG - Processing node 69d3731a2214054febe38488: baseline-arm64-baylibre 2026-04-08 08:47:35,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3731a2214054febe38488 2026-04-08 08:47:35,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:35,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3731a2214054febe38488 HTTP/1.1" 200 None 2026-04-08 08:47:35,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:35,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:35,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:35,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3731a2214054febe38488 HTTP/1.1" 200 None 2026-04-08 08:47:35,303 - lava_callback - ERROR - Error processing callback for node 69d3731a2214054febe38488: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34546 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:47:38,124 - lava_callback - INFO - Received callback for node 69d615d64b6493d8616623f3 INFO: 172.20.0.1:34562 - "POST /node/69d615d64b6493d8616623f3 HTTP/1.0" 202 Accepted 2026-04-08 08:47:38,130 - lava_callback - INFO - Starting async job submit for node 69d615d64b6493d8616623f3 2026-04-08 08:47:38,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:38,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d64b6493d8616623f3 HTTP/1.1" 200 None 2026-04-08 08:47:38,167 - lava_callback - DEBUG - Processing node 69d615d64b6493d8616623f3: baseline-arm64 2026-04-08 08:47:38,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:38,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:38,557 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d64b6493d8616623f3/log.txt.gz 2026-04-08 08:47:38,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:38,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:38,955 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d64b6493d8616623f3/lava-logs.yaml 2026-04-08 08:47:38,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34578 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:47:39,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:39,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d64b6493d8616623f3/lava_callback.json.gz 2026-04-08 08:47:39,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:39,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d64b6493d8616623f3 HTTP/1.1" 200 None 2026-04-08 08:47:39,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:39,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:39,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:39,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615d64b6493d8616623f3 HTTP/1.1" 200 None 2026-04-08 08:47:39,683 - lava_callback - INFO - Completed processing callback for node 69d615d64b6493d8616623f3 2026-04-08 08:47:45,393 - lava_callback - INFO - Received callback for node 69d615c14b6493d861662382 INFO: 172.20.0.1:36612 - "POST /node/69d615c14b6493d861662382 HTTP/1.0" 202 Accepted 2026-04-08 08:47:45,400 - lava_callback - INFO - Starting async job submit for node 69d615c14b6493d861662382 2026-04-08 08:47:45,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:45,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c14b6493d861662382 HTTP/1.1" 200 None 2026-04-08 08:47:45,436 - lava_callback - DEBUG - Processing node 69d615c14b6493d861662382: baseline-arm64 2026-04-08 08:47:45,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:45,811 - lava_callback - INFO - Received callback for node 69d615dc4b6493d861662403 INFO: 172.20.0.1:36628 - "POST /node/69d615dc4b6493d861662403 HTTP/1.0" 202 Accepted 2026-04-08 08:47:45,818 - lava_callback - INFO - Starting async job submit for node 69d615dc4b6493d861662403 2026-04-08 08:47:45,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:45,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:45,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c14b6493d861662382/log.txt.gz 2026-04-08 08:47:45,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615dc4b6493d861662403 HTTP/1.1" 200 None 2026-04-08 08:47:45,920 - lava_callback - DEBUG - Processing node 69d615dc4b6493d861662403: baseline-nfs-arm64 2026-04-08 08:47:46,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:46,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:46,465 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c14b6493d861662382/lava-logs.yaml 2026-04-08 08:47:46,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:46,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36638 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:47:46,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:46,615 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615dc4b6493d861662403/log.txt.gz 2026-04-08 08:47:46,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:46,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c14b6493d861662382/lava_callback.json.gz 2026-04-08 08:47:46,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:46,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c14b6493d861662382 HTTP/1.1" 200 None 2026-04-08 08:47:46,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:46,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:46,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:46,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:46,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c14b6493d861662382 HTTP/1.1" 200 None 2026-04-08 08:47:46,940 - lava_callback - INFO - Completed processing callback for node 69d615c14b6493d861662382 2026-04-08 08:47:47,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:47,226 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615dc4b6493d861662403/lava-logs.yaml 2026-04-08 08:47:47,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:47,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:47,399 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615dc4b6493d861662403/lava_callback.json.gz 2026-04-08 08:47:47,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:47,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615dc4b6493d861662403 HTTP/1.1" 200 None 2026-04-08 08:47:47,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:47,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:47,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:47,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615dc4b6493d861662403 HTTP/1.1" 200 None 2026-04-08 08:47:47,478 - lava_callback - INFO - Completed processing callback for node 69d615dc4b6493d861662403 2026-04-08 08:47:48,222 - lava_callback - INFO - Received callback for node 69d615d34b6493d8616623ec INFO: 172.20.0.1:36640 - "POST /node/69d615d34b6493d8616623ec HTTP/1.0" 202 Accepted 2026-04-08 08:47:48,228 - lava_callback - INFO - Starting async job submit for node 69d615d34b6493d8616623ec 2026-04-08 08:47:48,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:48,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d34b6493d8616623ec HTTP/1.1" 200 None 2026-04-08 08:47:48,265 - lava_callback - DEBUG - Processing node 69d615d34b6493d8616623ec: baseline-arm64 2026-04-08 08:47:48,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:48,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:48,716 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d34b6493d8616623ec/log.txt.gz 2026-04-08 08:47:48,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:49,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:49,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d34b6493d8616623ec/lava-logs.yaml 2026-04-08 08:47:49,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:47:49,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:47:49,398 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d34b6493d8616623ec/lava_callback.json.gz 2026-04-08 08:47:49,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:49,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d34b6493d8616623ec HTTP/1.1" 200 None 2026-04-08 08:47:49,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:49,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:47:49,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:49,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615d34b6493d8616623ec HTTP/1.1" 200 None 2026-04-08 08:47:49,495 - lava_callback - INFO - Completed processing callback for node 69d615d34b6493d8616623ec 2026-04-08 08:47:50,513 - lava_callback - INFO - Received callback for node 69d373272214054febe384c8 INFO: 172.20.0.1:42900 - "POST /node/69d373272214054febe384c8 HTTP/1.0" 202 Accepted 2026-04-08 08:47:50,520 - lava_callback - INFO - Starting async job submit for node 69d373272214054febe384c8 2026-04-08 08:47:50,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:50,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373272214054febe384c8 HTTP/1.1" 200 None 2026-04-08 08:47:50,537 - lava_callback - DEBUG - Processing node 69d373272214054febe384c8: baseline-arm64-clabbe 2026-04-08 08:47:50,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d373272214054febe384c8 2026-04-08 08:47:50,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:50,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373272214054febe384c8 HTTP/1.1" 200 None 2026-04-08 08:47:50,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:50,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:50,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:50,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d373272214054febe384c8 HTTP/1.1" 200 None 2026-04-08 08:47:50,599 - lava_callback - ERROR - Error processing callback for node 69d373272214054febe384c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:51,224 - lava_callback - INFO - Received callback for node 69d373292214054febe384cc INFO: 172.20.0.1:42914 - "POST /node/69d373292214054febe384cc HTTP/1.0" 202 Accepted 2026-04-08 08:47:51,232 - lava_callback - INFO - Starting async job submit for node 69d373292214054febe384cc 2026-04-08 08:47:51,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:51,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373292214054febe384cc HTTP/1.1" 200 None 2026-04-08 08:47:51,252 - lava_callback - DEBUG - Processing node 69d373292214054febe384cc: baseline-arm64-clabbe 2026-04-08 08:47:51,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d373292214054febe384cc 2026-04-08 08:47:51,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:51,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d373292214054febe384cc HTTP/1.1" 200 None 2026-04-08 08:47:51,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:51,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:51,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:51,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d373292214054febe384cc HTTP/1.1" 200 None 2026-04-08 08:47:51,310 - lava_callback - ERROR - Error processing callback for node 69d373292214054febe384cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:51,988 - lava_callback - INFO - Received callback for node 69d3732a2214054febe384d3 INFO: 172.20.0.1:42918 - "POST /node/69d3732a2214054febe384d3 HTTP/1.0" 202 Accepted 2026-04-08 08:47:51,997 - lava_callback - INFO - Starting async job submit for node 69d3732a2214054febe384d3 2026-04-08 08:47:51,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732a2214054febe384d3 HTTP/1.1" 200 None 2026-04-08 08:47:52,014 - lava_callback - DEBUG - Processing node 69d3732a2214054febe384d3: baseline-arm64-clabbe 2026-04-08 08:47:52,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3732a2214054febe384d3 2026-04-08 08:47:52,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732a2214054febe384d3 HTTP/1.1" 200 None 2026-04-08 08:47:52,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:52,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3732a2214054febe384d3 HTTP/1.1" 200 None 2026-04-08 08:47:52,108 - lava_callback - ERROR - Error processing callback for node 69d3732a2214054febe384d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:52,691 - lava_callback - INFO - Received callback for node 69d3732a2214054febe384d6 INFO: 172.20.0.1:42928 - "POST /node/69d3732a2214054febe384d6 HTTP/1.0" 202 Accepted 2026-04-08 08:47:52,697 - lava_callback - INFO - Starting async job submit for node 69d3732a2214054febe384d6 2026-04-08 08:47:52,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732a2214054febe384d6 HTTP/1.1" 200 None 2026-04-08 08:47:52,714 - lava_callback - DEBUG - Processing node 69d3732a2214054febe384d6: baseline-arm64-clabbe 2026-04-08 08:47:52,714 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3732a2214054febe384d6 2026-04-08 08:47:52,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732a2214054febe384d6 HTTP/1.1" 200 None 2026-04-08 08:47:52,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:52,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:52,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3732a2214054febe384d6 HTTP/1.1" 200 None 2026-04-08 08:47:52,774 - lava_callback - ERROR - Error processing callback for node 69d3732a2214054febe384d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:53,376 - lava_callback - INFO - Received callback for node 69d3732b2214054febe384db INFO: 172.20.0.1:42934 - "POST /node/69d3732b2214054febe384db HTTP/1.0" 202 Accepted 2026-04-08 08:47:53,382 - lava_callback - INFO - Starting async job submit for node 69d3732b2214054febe384db 2026-04-08 08:47:53,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:53,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732b2214054febe384db HTTP/1.1" 200 None 2026-04-08 08:47:53,402 - lava_callback - DEBUG - Processing node 69d3732b2214054febe384db: baseline-arm64-clabbe 2026-04-08 08:47:53,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3732b2214054febe384db 2026-04-08 08:47:53,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:53,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732b2214054febe384db HTTP/1.1" 200 None 2026-04-08 08:47:53,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:53,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:53,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:53,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3732b2214054febe384db HTTP/1.1" 200 None 2026-04-08 08:47:53,481 - lava_callback - ERROR - Error processing callback for node 69d3732b2214054febe384db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:54,026 - lava_callback - INFO - Received callback for node 69d3732d2214054febe384e1 INFO: 172.20.0.1:42940 - "POST /node/69d3732d2214054febe384e1 HTTP/1.0" 202 Accepted 2026-04-08 08:47:54,032 - lava_callback - INFO - Starting async job submit for node 69d3732d2214054febe384e1 2026-04-08 08:47:54,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732d2214054febe384e1 HTTP/1.1" 200 None 2026-04-08 08:47:54,047 - lava_callback - DEBUG - Processing node 69d3732d2214054febe384e1: baseline-arm64-clabbe 2026-04-08 08:47:54,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3732d2214054febe384e1 2026-04-08 08:47:54,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732d2214054febe384e1 HTTP/1.1" 200 None 2026-04-08 08:47:54,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:54,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3732d2214054febe384e1 HTTP/1.1" 200 None 2026-04-08 08:47:54,107 - lava_callback - ERROR - Error processing callback for node 69d3732d2214054febe384e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:47:54,706 - lava_callback - INFO - Received callback for node 69d3732c2214054febe384de INFO: 172.20.0.1:42954 - "POST /node/69d3732c2214054febe384de HTTP/1.0" 202 Accepted 2026-04-08 08:47:54,713 - lava_callback - INFO - Starting async job submit for node 69d3732c2214054febe384de 2026-04-08 08:47:54,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732c2214054febe384de HTTP/1.1" 200 None 2026-04-08 08:47:54,728 - lava_callback - DEBUG - Processing node 69d3732c2214054febe384de: baseline-arm64-clabbe 2026-04-08 08:47:54,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3732c2214054febe384de 2026-04-08 08:47:54,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3732c2214054febe384de HTTP/1.1" 200 None 2026-04-08 08:47:54,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d36c2e2214054febe37d71 HTTP/1.1" 200 None 2026-04-08 08:47:54,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:47:54,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3732c2214054febe384de HTTP/1.1" 200 None 2026-04-08 08:47:54,780 - lava_callback - ERROR - Error processing callback for node 69d3732c2214054febe384de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42966 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:48:04,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:04,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-08 08:48:06,508 - lava_callback - INFO - Received callback for node 69d615e84b6493d861662423 INFO: 172.20.0.1:44324 - "POST /node/69d615e84b6493d861662423 HTTP/1.0" 202 Accepted 2026-04-08 08:48:06,514 - lava_callback - INFO - Starting async job submit for node 69d615e84b6493d861662423 INFO: 172.20.0.1:44328 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:48:06,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:06,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e84b6493d861662423 HTTP/1.1" 200 None 2026-04-08 08:48:06,555 - lava_callback - DEBUG - Processing node 69d615e84b6493d861662423: baseline-nfs-arm64 2026-04-08 08:48:06,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:07,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:07,100 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e84b6493d861662423/log.txt.gz 2026-04-08 08:48:07,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:07,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:07,538 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e84b6493d861662423/lava-logs.yaml 2026-04-08 08:48:07,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:07,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:07,714 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e84b6493d861662423/lava_callback.json.gz 2026-04-08 08:48:07,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:07,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e84b6493d861662423 HTTP/1.1" 200 None 2026-04-08 08:48:07,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:07,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:07,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:07,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615e84b6493d861662423 HTTP/1.1" 200 None 2026-04-08 08:48:07,819 - lava_callback - INFO - Completed processing callback for node 69d615e84b6493d861662423 2026-04-08 08:48:08,314 - lava_callback - INFO - Received callback for node 69d615e64b6493d86166241c INFO: 172.20.0.1:44342 - "POST /node/69d615e64b6493d86166241c HTTP/1.0" 202 Accepted 2026-04-08 08:48:08,321 - lava_callback - INFO - Starting async job submit for node 69d615e64b6493d86166241c 2026-04-08 08:48:08,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:08,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e64b6493d86166241c HTTP/1.1" 200 None 2026-04-08 08:48:08,360 - lava_callback - DEBUG - Processing node 69d615e64b6493d86166241c: baseline-nfs-arm64 2026-04-08 08:48:08,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:08,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:08,857 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e64b6493d86166241c/log.txt.gz 2026-04-08 08:48:09,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:09,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:09,335 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e64b6493d86166241c/lava-logs.yaml 2026-04-08 08:48:09,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:09,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:09,552 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e64b6493d86166241c/lava_callback.json.gz 2026-04-08 08:48:09,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:09,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e64b6493d86166241c HTTP/1.1" 200 None 2026-04-08 08:48:09,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:09,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:09,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:09,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615e64b6493d86166241c HTTP/1.1" 200 None 2026-04-08 08:48:09,645 - lava_callback - INFO - Completed processing callback for node 69d615e64b6493d86166241c 2026-04-08 08:48:14,223 - lava_callback - INFO - Received callback for node 69d616194b6493d861662504 INFO: 172.20.0.1:57764 - "POST /node/69d616194b6493d861662504 HTTP/1.0" 202 Accepted 2026-04-08 08:48:14,230 - lava_callback - INFO - Starting async job submit for node 69d616194b6493d861662504 2026-04-08 08:48:14,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:14,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616194b6493d861662504 HTTP/1.1" 200 None 2026-04-08 08:48:14,285 - lava_callback - DEBUG - Processing node 69d616194b6493d861662504: baseline-arm64-foundriesio 2026-04-08 08:48:14,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:14,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:14,674 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d616194b6493d861662504/log.txt.gz 2026-04-08 08:48:14,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:14,866 - lava_callback - INFO - Received callback for node 69d6161a4b6493d86166250b INFO: 172.20.0.1:57768 - "POST /node/69d6161a4b6493d86166250b HTTP/1.0" 202 Accepted 2026-04-08 08:48:14,877 - lava_callback - INFO - Starting async job submit for node 69d6161a4b6493d86166250b 2026-04-08 08:48:14,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:14,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6161a4b6493d86166250b HTTP/1.1" 200 None 2026-04-08 08:48:14,929 - lava_callback - DEBUG - Processing node 69d6161a4b6493d86166250b: baseline-arm64-foundriesio 2026-04-08 08:48:15,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:15,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d616194b6493d861662504/lava-logs.yaml 2026-04-08 08:48:15,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57774 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 08:48:15,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:15,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:15,396 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d616194b6493d861662504/lava_callback.json.gz 2026-04-08 08:48:15,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:15,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616194b6493d861662504 HTTP/1.1" 200 None 2026-04-08 08:48:15,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:15,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:15,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:15,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:15,466 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d6161a4b6493d86166250b/log.txt.gz 2026-04-08 08:48:15,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616194b6493d861662504 HTTP/1.1" 200 None 2026-04-08 08:48:15,577 - lava_callback - INFO - Completed processing callback for node 69d616194b6493d861662504 2026-04-08 08:48:15,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:16,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:16,003 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d6161a4b6493d86166250b/lava-logs.yaml 2026-04-08 08:48:16,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:16,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:16,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d6161a4b6493d86166250b/lava_callback.json.gz 2026-04-08 08:48:16,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:16,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6161a4b6493d86166250b HTTP/1.1" 200 None 2026-04-08 08:48:16,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:16,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:16,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:16,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6161a4b6493d86166250b HTTP/1.1" 200 None 2026-04-08 08:48:16,266 - lava_callback - INFO - Completed processing callback for node 69d6161a4b6493d86166250b INFO: 172.20.0.1:57790 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:48:18,274 - lava_callback - INFO - Received callback for node 69d616164b6493d8616624fa INFO: 172.20.0.1:57804 - "POST /node/69d616164b6493d8616624fa HTTP/1.0" 202 Accepted 2026-04-08 08:48:18,280 - lava_callback - INFO - Starting async job submit for node 69d616164b6493d8616624fa 2026-04-08 08:48:18,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:18,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616164b6493d8616624fa HTTP/1.1" 200 None 2026-04-08 08:48:18,315 - lava_callback - DEBUG - Processing node 69d616164b6493d8616624fa: baseline-arm64-pengutronix 2026-04-08 08:48:18,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:18,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:18,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d616164b6493d8616624fa/log.txt.gz 2026-04-08 08:48:18,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:18,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:18,893 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d616164b6493d8616624fa/lava-logs.yaml 2026-04-08 08:48:18,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:19,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:19,035 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d616164b6493d8616624fa/lava_callback.json.gz 2026-04-08 08:48:19,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:19,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616164b6493d8616624fa HTTP/1.1" 200 None 2026-04-08 08:48:19,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:19,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:19,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:19,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616164b6493d8616624fa HTTP/1.1" 200 None 2026-04-08 08:48:19,117 - lava_callback - INFO - Completed processing callback for node 69d616164b6493d8616624fa 2026-04-08 08:48:26,148 - lava_callback - INFO - Received callback for node 69d616174b6493d861662501 INFO: 172.20.0.1:52578 - "POST /node/69d616174b6493d861662501 HTTP/1.0" 202 Accepted 2026-04-08 08:48:26,155 - lava_callback - INFO - Starting async job submit for node 69d616174b6493d861662501 2026-04-08 08:48:26,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:26,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616174b6493d861662501 HTTP/1.1" 200 None 2026-04-08 08:48:26,201 - lava_callback - DEBUG - Processing node 69d616174b6493d861662501: baseline-arm64-pengutronix 2026-04-08 08:48:26,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:26,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:26,532 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d616174b6493d861662501/log.txt.gz INFO: 172.20.0.1:52580 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:48:26,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:26,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:26,816 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d616174b6493d861662501/lava-logs.yaml 2026-04-08 08:48:26,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:27,133 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:27,135 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d616174b6493d861662501/lava_callback.json.gz 2026-04-08 08:48:27,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:27,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616174b6493d861662501 HTTP/1.1" 200 None 2026-04-08 08:48:27,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:27,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:27,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:27,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616174b6493d861662501 HTTP/1.1" 200 None 2026-04-08 08:48:27,219 - lava_callback - INFO - Completed processing callback for node 69d616174b6493d861662501 2026-04-08 08:48:34,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:34,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43124 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:48:45,555 - lava_callback - INFO - Received callback for node 69d616124b6493d8616624f4 INFO: 172.20.0.1:56208 - "POST /node/69d616124b6493d8616624f4 HTTP/1.0" 202 Accepted 2026-04-08 08:48:45,561 - lava_callback - INFO - Starting async job submit for node 69d616124b6493d8616624f4 2026-04-08 08:48:45,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:45,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616124b6493d8616624f4 HTTP/1.1" 200 None 2026-04-08 08:48:45,599 - lava_callback - DEBUG - Processing node 69d616124b6493d8616624f4: baseline-arm64-kontron 2026-04-08 08:48:45,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:45,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:45,947 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d616124b6493d8616624f4/log.txt.gz 2026-04-08 08:48:46,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:46,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:46,258 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d616124b6493d8616624f4/lava-logs.yaml 2026-04-08 08:48:46,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:46,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:46,439 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d616124b6493d8616624f4/lava_callback.json.gz 2026-04-08 08:48:46,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:46,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616124b6493d8616624f4 HTTP/1.1" 200 None 2026-04-08 08:48:46,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:46,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:46,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:48:46,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616124b6493d8616624f4 HTTP/1.1" 200 None 2026-04-08 08:48:46,552 - lava_callback - INFO - Completed processing callback for node 69d616124b6493d8616624f4 2026-04-08 08:48:46,835 - lava_callback - INFO - Received callback for node 69d616144b6493d8616624f7 INFO: 172.20.0.1:56222 - "POST /node/69d616144b6493d8616624f7 HTTP/1.0" 202 Accepted 2026-04-08 08:48:46,842 - lava_callback - INFO - Starting async job submit for node 69d616144b6493d8616624f7 2026-04-08 08:48:46,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:46,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616144b6493d8616624f7 HTTP/1.1" 200 None 2026-04-08 08:48:46,883 - lava_callback - DEBUG - Processing node 69d616144b6493d8616624f7: baseline-arm64-kontron 2026-04-08 08:48:47,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:47,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:47,332 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d616144b6493d8616624f7/log.txt.gz 2026-04-08 08:48:47,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:47,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:47,692 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d616144b6493d8616624f7/lava-logs.yaml 2026-04-08 08:48:47,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:47,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:47,872 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d616144b6493d8616624f7/lava_callback.json.gz 2026-04-08 08:48:47,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:47,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616144b6493d8616624f7 HTTP/1.1" 200 None 2026-04-08 08:48:47,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:47,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:47,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:47,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616144b6493d8616624f7 HTTP/1.1" 200 None 2026-04-08 08:48:47,955 - lava_callback - INFO - Completed processing callback for node 69d616144b6493d8616624f7 2026-04-08 08:48:53,061 - lava_callback - INFO - Received callback for node 69d615fd4b6493d861662478 INFO: 172.20.0.1:47926 - "POST /node/69d615fd4b6493d861662478 HTTP/1.0" 202 Accepted 2026-04-08 08:48:53,067 - lava_callback - INFO - Starting async job submit for node 69d615fd4b6493d861662478 2026-04-08 08:48:53,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:53,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615fd4b6493d861662478 HTTP/1.1" 200 None 2026-04-08 08:48:53,113 - lava_callback - DEBUG - Processing node 69d615fd4b6493d861662478: baseline-arm64-broonie 2026-04-08 08:48:53,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:53,486 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:53,488 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615fd4b6493d861662478/log.txt.gz 2026-04-08 08:48:53,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:53,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:53,875 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615fd4b6493d861662478/lava-logs.yaml 2026-04-08 08:48:53,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:54,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:54,029 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615fd4b6493d861662478/lava_callback.json.gz 2026-04-08 08:48:54,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:54,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615fd4b6493d861662478 HTTP/1.1" 200 None 2026-04-08 08:48:54,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:54,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:54,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:54,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615fd4b6493d861662478 HTTP/1.1" 200 None 2026-04-08 08:48:54,114 - lava_callback - INFO - Completed processing callback for node 69d615fd4b6493d861662478 2026-04-08 08:48:54,811 - lava_callback - INFO - Received callback for node 69d615ff4b6493d86166247b INFO: 172.20.0.1:47942 - "POST /node/69d615ff4b6493d86166247b HTTP/1.0" 202 Accepted 2026-04-08 08:48:54,817 - lava_callback - INFO - Starting async job submit for node 69d615ff4b6493d86166247b 2026-04-08 08:48:54,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:54,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615ff4b6493d86166247b HTTP/1.1" 200 None 2026-04-08 08:48:54,855 - lava_callback - DEBUG - Processing node 69d615ff4b6493d86166247b: baseline-arm64-broonie 2026-04-08 08:48:55,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:55,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:55,228 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615ff4b6493d86166247b/log.txt.gz 2026-04-08 08:48:55,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:55,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:55,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615ff4b6493d86166247b/lava-logs.yaml 2026-04-08 08:48:55,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:55,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:55,728 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615ff4b6493d86166247b/lava_callback.json.gz 2026-04-08 08:48:55,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:55,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615ff4b6493d86166247b HTTP/1.1" 200 None 2026-04-08 08:48:55,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:55,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:55,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:55,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615ff4b6493d86166247b HTTP/1.1" 200 None 2026-04-08 08:48:55,812 - lava_callback - INFO - Completed processing callback for node 69d615ff4b6493d86166247b 2026-04-08 08:48:56,216 - lava_callback - INFO - Received callback for node 69d616034b6493d861662486 INFO: 172.20.0.1:47958 - "POST /node/69d616034b6493d861662486 HTTP/1.0" 202 Accepted 2026-04-08 08:48:56,223 - lava_callback - INFO - Starting async job submit for node 69d616034b6493d861662486 2026-04-08 08:48:56,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:56,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616034b6493d861662486 HTTP/1.1" 200 None 2026-04-08 08:48:56,262 - lava_callback - DEBUG - Processing node 69d616034b6493d861662486: baseline-arm64-broonie INFO: 172.20.0.1:47968 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:48:56,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:56,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:56,646 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616034b6493d861662486/log.txt.gz 2026-04-08 08:48:56,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:56,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:56,985 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616034b6493d861662486/lava-logs.yaml 2026-04-08 08:48:56,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:57,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:57,120 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616034b6493d861662486/lava_callback.json.gz 2026-04-08 08:48:57,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:57,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616034b6493d861662486 HTTP/1.1" 200 None 2026-04-08 08:48:57,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:57,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:57,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:57,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616034b6493d861662486 HTTP/1.1" 200 None 2026-04-08 08:48:57,205 - lava_callback - INFO - Completed processing callback for node 69d616034b6493d861662486 2026-04-08 08:48:57,649 - lava_callback - INFO - Received callback for node 69d616084b6493d8616624c7 INFO: 172.20.0.1:47984 - "POST /node/69d616084b6493d8616624c7 HTTP/1.0" 202 Accepted 2026-04-08 08:48:57,656 - lava_callback - INFO - Starting async job submit for node 69d616084b6493d8616624c7 2026-04-08 08:48:57,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:57,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616084b6493d8616624c7 HTTP/1.1" 200 None 2026-04-08 08:48:57,694 - lava_callback - DEBUG - Processing node 69d616084b6493d8616624c7: baseline-arm64-broonie 2026-04-08 08:48:57,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:58,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:58,072 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616084b6493d8616624c7/log.txt.gz 2026-04-08 08:48:58,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:58,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:58,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616084b6493d8616624c7/lava-logs.yaml 2026-04-08 08:48:58,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:48:58,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:48:58,537 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616084b6493d8616624c7/lava_callback.json.gz 2026-04-08 08:48:58,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:58,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616084b6493d8616624c7 HTTP/1.1" 200 None 2026-04-08 08:48:58,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:58,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:48:58,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:48:58,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616084b6493d8616624c7 HTTP/1.1" 200 None 2026-04-08 08:48:58,621 - lava_callback - INFO - Completed processing callback for node 69d616084b6493d8616624c7 2026-04-08 08:49:04,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:05,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:34280 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:49:07,150 - lava_callback - INFO - Received callback for node 69d615e34b6493d861662416 INFO: 172.20.0.1:34284 - "POST /node/69d615e34b6493d861662416 HTTP/1.0" 202 Accepted 2026-04-08 08:49:07,157 - lava_callback - INFO - Starting async job submit for node 69d615e34b6493d861662416 2026-04-08 08:49:07,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e34b6493d861662416 HTTP/1.1" 200 None 2026-04-08 08:49:07,200 - lava_callback - DEBUG - Processing node 69d615e34b6493d861662416: baseline-nfs-arm64 2026-04-08 08:49:07,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:07,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:07,657 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e34b6493d861662416/log.txt.gz 2026-04-08 08:49:07,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:08,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:08,218 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e34b6493d861662416/lava-logs.yaml 2026-04-08 08:49:08,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:08,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:08,420 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e34b6493d861662416/lava_callback.json.gz 2026-04-08 08:49:08,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:08,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e34b6493d861662416 HTTP/1.1" 200 None 2026-04-08 08:49:08,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:08,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:49:08,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:08,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615e34b6493d861662416 HTTP/1.1" 200 None 2026-04-08 08:49:08,500 - lava_callback - INFO - Completed processing callback for node 69d615e34b6493d861662416 2026-04-08 08:49:14,486 - lava_callback - INFO - Received callback for node 69d615f74b6493d861662473 INFO: 172.20.0.1:45912 - "POST /node/69d615f74b6493d861662473 HTTP/1.0" 202 Accepted 2026-04-08 08:49:14,492 - lava_callback - INFO - Starting async job submit for node 69d615f74b6493d861662473 2026-04-08 08:49:14,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:14,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615f74b6493d861662473 HTTP/1.1" 200 None 2026-04-08 08:49:14,537 - lava_callback - DEBUG - Processing node 69d615f74b6493d861662473: baseline-arm64-broonie 2026-04-08 08:49:14,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:14,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:14,881 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615f74b6493d861662473/log.txt.gz 2026-04-08 08:49:15,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45924 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 08:49:15,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:15,214 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615f74b6493d861662473/lava-logs.yaml 2026-04-08 08:49:15,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:15,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:15,361 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d615f74b6493d861662473/lava_callback.json.gz 2026-04-08 08:49:15,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:15,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615f74b6493d861662473 HTTP/1.1" 200 None 2026-04-08 08:49:15,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:15,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:49:15,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:15,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615f74b6493d861662473 HTTP/1.1" 200 None 2026-04-08 08:49:15,465 - lava_callback - INFO - Completed processing callback for node 69d615f74b6493d861662473 INFO: 172.20.0.1:45936 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:49:17,702 - lava_callback - INFO - Received callback for node 69d615c84b6493d861662399 INFO: 172.20.0.1:45952 - "POST /node/69d615c84b6493d861662399 HTTP/1.0" 202 Accepted 2026-04-08 08:49:17,708 - lava_callback - INFO - Starting async job submit for node 69d615c84b6493d861662399 2026-04-08 08:49:17,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:17,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c84b6493d861662399 HTTP/1.1" 200 None 2026-04-08 08:49:17,747 - lava_callback - DEBUG - Processing node 69d615c84b6493d861662399: baseline-arm64 2026-04-08 08:49:18,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:18,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:18,225 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c84b6493d861662399/log.txt.gz 2026-04-08 08:49:18,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:18,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:18,612 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c84b6493d861662399/lava-logs.yaml 2026-04-08 08:49:18,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:18,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:18,751 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615c84b6493d861662399/lava_callback.json.gz 2026-04-08 08:49:18,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:18,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615c84b6493d861662399 HTTP/1.1" 200 None 2026-04-08 08:49:18,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:18,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:49:18,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:18,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615c84b6493d861662399 HTTP/1.1" 200 None 2026-04-08 08:49:18,839 - lava_callback - INFO - Completed processing callback for node 69d615c84b6493d861662399 2026-04-08 08:49:19,983 - lava_callback - INFO - Received callback for node 69d615d94b6493d8616623fd INFO: 172.20.0.1:55302 - "POST /node/69d615d94b6493d8616623fd HTTP/1.0" 202 Accepted 2026-04-08 08:49:19,990 - lava_callback - INFO - Starting async job submit for node 69d615d94b6493d8616623fd 2026-04-08 08:49:20,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:20,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d94b6493d8616623fd HTTP/1.1" 200 None 2026-04-08 08:49:20,027 - lava_callback - DEBUG - Processing node 69d615d94b6493d8616623fd: baseline-nfs-arm64 2026-04-08 08:49:20,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:20,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:20,453 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615d94b6493d8616623fd/log.txt.gz 2026-04-08 08:49:20,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:20,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:20,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615d94b6493d8616623fd/lava-logs.yaml 2026-04-08 08:49:20,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:21,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:21,013 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615d94b6493d8616623fd/lava_callback.json.gz 2026-04-08 08:49:21,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:21,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d94b6493d8616623fd HTTP/1.1" 200 None 2026-04-08 08:49:21,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:21,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:49:21,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:21,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615d94b6493d8616623fd HTTP/1.1" 200 None 2026-04-08 08:49:21,106 - lava_callback - INFO - Completed processing callback for node 69d615d94b6493d8616623fd INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:49:35,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:35,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58432 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:49:49,683 - lava_callback - INFO - Received callback for node 69d615ce4b6493d8616623dc INFO: 172.20.0.1:58438 - "POST /node/69d615ce4b6493d8616623dc HTTP/1.0" 202 Accepted 2026-04-08 08:49:49,691 - lava_callback - INFO - Starting async job submit for node 69d615ce4b6493d8616623dc 2026-04-08 08:49:49,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:49,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615ce4b6493d8616623dc HTTP/1.1" 200 None 2026-04-08 08:49:49,747 - lava_callback - DEBUG - Processing node 69d615ce4b6493d8616623dc: baseline-arm64 2026-04-08 08:49:50,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:51,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:51,313 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615ce4b6493d8616623dc/log.txt.gz 2026-04-08 08:49:51,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:52,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:52,762 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615ce4b6493d8616623dc/lava-logs.yaml 2026-04-08 08:49:52,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:53,028 - lava_callback - INFO - Received callback for node 69d614e84b6493d861662222 INFO: 172.20.0.1:37884 - "POST /node/69d614e84b6493d861662222 HTTP/1.0" 202 Accepted 2026-04-08 08:49:53,037 - lava_callback - INFO - Starting async job submit for node 69d614e84b6493d861662222 2026-04-08 08:49:53,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:53,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e84b6493d861662222 HTTP/1.1" 200 None 2026-04-08 08:49:53,088 - lava_callback - DEBUG - Processing node 69d614e84b6493d861662222: ltp-syscalls-ipc 2026-04-08 08:49:53,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:53,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:53,939 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615ce4b6493d8616623dc/lava_callback.json.gz 2026-04-08 08:49:53,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:53,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615ce4b6493d8616623dc HTTP/1.1" 200 None 2026-04-08 08:49:53,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:54,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:49:54,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:54,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615ce4b6493d8616623dc HTTP/1.1" 200 None 2026-04-08 08:49:54,040 - lava_callback - INFO - Completed processing callback for node 69d615ce4b6493d8616623dc 2026-04-08 08:49:54,371 - lava_callback - INFO - Received callback for node 69d614e44b6493d861662217 INFO: 172.20.0.1:37892 - "POST /node/69d614e44b6493d861662217 HTTP/1.0" 202 Accepted 2026-04-08 08:49:54,381 - lava_callback - INFO - Starting async job submit for node 69d614e44b6493d861662217 2026-04-08 08:49:54,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:54,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e44b6493d861662217 HTTP/1.1" 200 None 2026-04-08 08:49:54,448 - lava_callback - DEBUG - Processing node 69d614e44b6493d861662217: ltp-sched 2026-04-08 08:49:54,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:54,854 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69d614e84b6493d861662222/log.txt.gz 2026-04-08 08:49:55,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:55,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:56,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:56,220 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69d614e44b6493d861662217/log.txt.gz 2026-04-08 08:49:56,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:56,442 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69d614e84b6493d861662222/lava-logs.yaml 2026-04-08 08:49:56,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:37904 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:49:56,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:57,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:57,688 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69d614e84b6493d861662222/lava_callback.json.gz 2026-04-08 08:49:57,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:57,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e84b6493d861662222 HTTP/1.1" 200 None 2026-04-08 08:49:57,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:57,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:49:57,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:57,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614e84b6493d861662222 HTTP/1.1" 200 None 2026-04-08 08:49:57,832 - lava_callback - INFO - Completed processing callback for node 69d614e84b6493d861662222 2026-04-08 08:49:57,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:57,881 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69d614e44b6493d861662217/lava-logs.yaml 2026-04-08 08:49:57,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:49:59,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:49:59,152 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69d614e44b6493d861662217/lava_callback.json.gz 2026-04-08 08:49:59,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:59,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e44b6493d861662217 HTTP/1.1" 200 None 2026-04-08 08:49:59,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:59,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:49:59,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:49:59,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614e44b6493d861662217 HTTP/1.1" 200 None 2026-04-08 08:49:59,292 - lava_callback - INFO - Completed processing callback for node 69d614e44b6493d861662217 2026-04-08 08:50:05,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:05,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:50:13,241 - lava_callback - INFO - Received callback for node 69d614d84b6493d8616621c7 INFO: 172.20.0.1:49812 - "POST /node/69d614d84b6493d8616621c7 HTTP/1.0" 202 Accepted 2026-04-08 08:50:13,249 - lava_callback - INFO - Starting async job submit for node 69d614d84b6493d8616621c7 2026-04-08 08:50:13,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:13,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614d84b6493d8616621c7 HTTP/1.1" 200 None 2026-04-08 08:50:13,316 - lava_callback - DEBUG - Processing node 69d614d84b6493d8616621c7: ltp-containers 2026-04-08 08:50:13,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:14,785 - lava_callback - INFO - Received callback for node 69d614e94b6493d861662225 INFO: 172.20.0.1:49828 - "POST /node/69d614e94b6493d861662225 HTTP/1.0" 202 Accepted 2026-04-08 08:50:14,796 - lava_callback - INFO - Starting async job submit for node 69d614e94b6493d861662225 2026-04-08 08:50:14,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:14,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e94b6493d861662225 HTTP/1.1" 200 None 2026-04-08 08:50:14,855 - lava_callback - DEBUG - Processing node 69d614e94b6493d861662225: ltp-timers 2026-04-08 08:50:15,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:15,058 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69d614d84b6493d8616621c7/log.txt.gz INFO: 172.20.0.1:49838 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 08:50:15,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:15,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49842 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:50:16,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:16,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69d614e94b6493d861662225/log.txt.gz 2026-04-08 08:50:16,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:16,966 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69d614d84b6493d8616621c7/lava-logs.yaml 2026-04-08 08:50:17,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:17,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:18,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:18,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69d614d84b6493d8616621c7/lava_callback.json.gz 2026-04-08 08:50:18,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:18,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614d84b6493d8616621c7 HTTP/1.1" 200 None 2026-04-08 08:50:18,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:18,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:50:18,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:18,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614d84b6493d8616621c7 HTTP/1.1" 200 None 2026-04-08 08:50:18,258 - lava_callback - INFO - Completed processing callback for node 69d614d84b6493d8616621c7 2026-04-08 08:50:18,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:18,387 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69d614e94b6493d861662225/lava-logs.yaml 2026-04-08 08:50:18,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:19,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:19,596 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69d614e94b6493d861662225/lava_callback.json.gz 2026-04-08 08:50:19,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:19,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614e94b6493d861662225 HTTP/1.1" 200 None 2026-04-08 08:50:19,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:19,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:50:19,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:19,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614e94b6493d861662225 HTTP/1.1" 200 None 2026-04-08 08:50:19,725 - lava_callback - INFO - Completed processing callback for node 69d614e94b6493d861662225 INFO: 172.20.0.1:34792 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:50:28,258 - lava_callback - INFO - Received callback for node 69d615db4b6493d861662400 INFO: 172.20.0.1:34808 - "POST /node/69d615db4b6493d861662400 HTTP/1.0" 202 Accepted 2026-04-08 08:50:28,269 - lava_callback - INFO - Starting async job submit for node 69d615db4b6493d861662400 2026-04-08 08:50:28,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:28,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615db4b6493d861662400 HTTP/1.1" 200 None 2026-04-08 08:50:28,318 - lava_callback - DEBUG - Processing node 69d615db4b6493d861662400: baseline-nfs-arm64 2026-04-08 08:50:28,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:30,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:30,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615db4b6493d861662400/log.txt.gz 2026-04-08 08:50:30,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:31,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:31,506 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615db4b6493d861662400/lava-logs.yaml 2026-04-08 08:50:31,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:32,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:32,522 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615db4b6493d861662400/lava_callback.json.gz 2026-04-08 08:50:32,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:32,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615db4b6493d861662400 HTTP/1.1" 200 None 2026-04-08 08:50:32,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:32,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:50:32,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:32,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615db4b6493d861662400 HTTP/1.1" 200 None 2026-04-08 08:50:32,649 - lava_callback - INFO - Completed processing callback for node 69d615db4b6493d861662400 2026-04-08 08:50:35,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:35,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43584 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:50:54,424 - lava_callback - INFO - Received callback for node 69d6126d4b6493d861662006 INFO: 172.20.0.1:38012 - "POST /node/69d6126d4b6493d861662006 HTTP/1.0" 202 Accepted 2026-04-08 08:50:54,436 - lava_callback - INFO - Starting async job submit for node 69d6126d4b6493d861662006 2026-04-08 08:50:54,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:54,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6126d4b6493d861662006 HTTP/1.1" 200 None 2026-04-08 08:50:54,558 - lava_callback - DEBUG - Processing node 69d6126d4b6493d861662006: ltp-smoke 2026-04-08 08:50:55,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:56,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:56,401 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69d6126d4b6493d861662006/log.txt.gz INFO: 172.20.0.1:38020 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:50:56,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:58,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:58,175 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69d6126d4b6493d861662006/lava-logs.yaml 2026-04-08 08:50:58,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:50:59,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:50:59,301 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69d6126d4b6493d861662006/lava_callback.json.gz 2026-04-08 08:50:59,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:59,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6126d4b6493d861662006 HTTP/1.1" 200 None 2026-04-08 08:50:59,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:59,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f134b6493d861661fa6 HTTP/1.1" 200 None 2026-04-08 08:50:59,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:50:59,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6126d4b6493d861662006 HTTP/1.1" 200 None 2026-04-08 08:50:59,740 - lava_callback - INFO - Completed processing callback for node 69d6126d4b6493d861662006 2026-04-08 08:51:05,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:05,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44142 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:51:32,926 - lava_callback - INFO - Received callback for node 69d615444b6493d8616622ca INFO: 172.20.0.1:48510 - "POST /node/69d615444b6493d8616622ca HTTP/1.0" 202 Accepted 2026-04-08 08:51:32,935 - lava_callback - INFO - Starting async job submit for node 69d615444b6493d8616622ca 2026-04-08 08:51:32,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:32,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615444b6493d8616622ca HTTP/1.1" 200 None 2026-04-08 08:51:32,994 - lava_callback - DEBUG - Processing node 69d615444b6493d8616622ca: baseline-arm-broonie 2026-04-08 08:51:33,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:51:34,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:51:34,299 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d615444b6493d8616622ca/log.txt.gz 2026-04-08 08:51:34,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:51:35,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:51:35,820 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d615444b6493d8616622ca/lava-logs.yaml 2026-04-08 08:51:35,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48516 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:51:36,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:51:36,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d615444b6493d8616622ca/lava_callback.json.gz 2026-04-08 08:51:36,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:37,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615444b6493d8616622ca HTTP/1.1" 200 None 2026-04-08 08:51:37,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:37,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:51:37,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:37,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615444b6493d8616622ca HTTP/1.1" 200 None 2026-04-08 08:51:37,074 - lava_callback - INFO - Completed processing callback for node 69d615444b6493d8616622ca INFO: 172.20.0.1:48530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:51:54,486 - lava_callback - INFO - Received callback for node 69d616054b6493d8616624bc INFO: 172.20.0.1:36804 - "POST /node/69d616054b6493d8616624bc HTTP/1.0" 202 Accepted 2026-04-08 08:51:54,494 - lava_callback - INFO - Starting async job submit for node 69d616054b6493d8616624bc 2026-04-08 08:51:54,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:54,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616054b6493d8616624bc HTTP/1.1" 200 None 2026-04-08 08:51:54,566 - lava_callback - DEBUG - Processing node 69d616054b6493d8616624bc: baseline-arm64-broonie 2026-04-08 08:51:55,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:51:56,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:51:56,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616054b6493d8616624bc/log.txt.gz 2026-04-08 08:51:56,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36818 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:51:57,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:51:57,570 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616054b6493d8616624bc/lava-logs.yaml 2026-04-08 08:51:57,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:51:58,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:51:58,622 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d616054b6493d8616624bc/lava_callback.json.gz 2026-04-08 08:51:58,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:58,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616054b6493d8616624bc HTTP/1.1" 200 None 2026-04-08 08:51:58,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:58,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:51:58,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:51:58,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616054b6493d8616624bc HTTP/1.1" 200 None 2026-04-08 08:51:58,860 - lava_callback - INFO - Completed processing callback for node 69d616054b6493d8616624bc 2026-04-08 08:52:05,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:05,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38618 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:52:12,178 - lava_callback - INFO - Received callback for node 69d3742c2214054febe3875b INFO: 172.20.0.1:38070 - "POST /node/69d3742c2214054febe3875b HTTP/1.0" 202 Accepted 2026-04-08 08:52:12,186 - lava_callback - INFO - Starting async job submit for node 69d3742c2214054febe3875b 2026-04-08 08:52:12,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:12,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3742c2214054febe3875b HTTP/1.1" 200 None 2026-04-08 08:52:12,207 - lava_callback - DEBUG - Processing node 69d3742c2214054febe3875b: baseline-arm-baylibre 2026-04-08 08:52:12,207 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3742c2214054febe3875b 2026-04-08 08:52:12,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:12,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3742c2214054febe3875b HTTP/1.1" 200 None 2026-04-08 08:52:12,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:12,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 08:52:12,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:12,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3742c2214054febe3875b HTTP/1.1" 200 None 2026-04-08 08:52:12,289 - lava_callback - ERROR - Error processing callback for node 69d3742c2214054febe3875b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45616 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:52:33,123 - lava_callback - INFO - Received callback for node 69d614b74b6493d861662139 INFO: 172.20.0.1:40668 - "POST /node/69d614b74b6493d861662139 HTTP/1.0" 202 Accepted 2026-04-08 08:52:33,129 - lava_callback - INFO - Starting async job submit for node 69d614b74b6493d861662139 2026-04-08 08:52:33,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:33,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614b74b6493d861662139 HTTP/1.1" 200 None 2026-04-08 08:52:33,167 - lava_callback - DEBUG - Processing node 69d614b74b6493d861662139: baseline-arm-broonie 2026-04-08 08:52:33,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:52:33,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:52:33,520 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b74b6493d861662139/log.txt.gz 2026-04-08 08:52:33,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:52:33,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:52:33,832 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b74b6493d861662139/lava-logs.yaml 2026-04-08 08:52:33,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:52:34,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:52:34,038 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d614b74b6493d861662139/lava_callback.json.gz 2026-04-08 08:52:34,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:34,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614b74b6493d861662139 HTTP/1.1" 200 None 2026-04-08 08:52:34,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:34,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:52:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:34,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614b74b6493d861662139 HTTP/1.1" 200 None 2026-04-08 08:52:34,120 - lava_callback - INFO - Completed processing callback for node 69d614b74b6493d861662139 2026-04-08 08:52:35,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:52:35,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:53:54,501 - lava_callback - INFO - Received callback for node 69d614ea4b6493d861662228 INFO: 172.20.0.1:51756 - "POST /node/69d614ea4b6493d861662228 HTTP/1.0" 202 Accepted 2026-04-08 08:53:54,507 - lava_callback - INFO - Starting async job submit for node 69d614ea4b6493d861662228 2026-04-08 08:53:54,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:53:54,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614ea4b6493d861662228 HTTP/1.1" 200 None 2026-04-08 08:53:54,554 - lava_callback - DEBUG - Processing node 69d614ea4b6493d861662228: ltp-watchqueue 2026-04-08 08:53:54,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:53:55,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:53:55,055 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69d614ea4b6493d861662228/log.txt.gz 2026-04-08 08:53:55,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:53:55,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:53:55,474 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69d614ea4b6493d861662228/lava-logs.yaml 2026-04-08 08:53:55,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:53:55,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:53:55,691 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69d614ea4b6493d861662228/lava_callback.json.gz 2026-04-08 08:53:55,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:53:55,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614ea4b6493d861662228 HTTP/1.1" 200 None 2026-04-08 08:53:55,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:53:55,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:53:55,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:53:55,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614ea4b6493d861662228 HTTP/1.1" 200 None 2026-04-08 08:53:55,782 - lava_callback - INFO - Completed processing callback for node 69d614ea4b6493d861662228 INFO: 172.20.0.1:51772 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:54:05,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:05,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-08 08:54:05,561 - lava_callback - INFO - Received callback for node 69d615d74b6493d8616623fa INFO: 172.20.0.1:57762 - "POST /node/69d615d74b6493d8616623fa HTTP/1.0" 202 Accepted 2026-04-08 08:54:05,568 - lava_callback - INFO - Starting async job submit for node 69d615d74b6493d8616623fa 2026-04-08 08:54:05,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:05,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d74b6493d8616623fa HTTP/1.1" 200 None 2026-04-08 08:54:05,620 - lava_callback - DEBUG - Processing node 69d615d74b6493d8616623fa: baseline-arm64 2026-04-08 08:54:06,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:06,134 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:06,136 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d74b6493d8616623fa/log.txt.gz 2026-04-08 08:54:06,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57766 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:54:06,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:06,542 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d74b6493d8616623fa/lava-logs.yaml 2026-04-08 08:54:06,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:06,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:06,708 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d74b6493d8616623fa/lava_callback.json.gz DEBUG: Setup failed for 69d615d74b6493d8616623fa. Transitting job node result: incomplete -> fail 2026-04-08 08:54:06,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:06,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d74b6493d8616623fa HTTP/1.1" 200 None 2026-04-08 08:54:06,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:06,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:54:06,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:06,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615d74b6493d8616623fa HTTP/1.1" 200 None 2026-04-08 08:54:06,938 - lava_callback - INFO - Completed processing callback for node 69d615d74b6493d8616623fa 2026-04-08 08:54:12,201 - lava_callback - INFO - Received callback for node 69d374a72214054febe38817 INFO: 172.20.0.1:40914 - "POST /node/69d374a72214054febe38817 HTTP/1.0" 202 Accepted 2026-04-08 08:54:12,206 - lava_callback - INFO - Starting async job submit for node 69d374a72214054febe38817 2026-04-08 08:54:12,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:12,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d374a72214054febe38817 HTTP/1.1" 200 None 2026-04-08 08:54:12,229 - lava_callback - DEBUG - Processing node 69d374a72214054febe38817: baseline-arm-baylibre 2026-04-08 08:54:12,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d374a72214054febe38817 2026-04-08 08:54:12,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:12,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d374a72214054febe38817 HTTP/1.1" 200 None 2026-04-08 08:54:12,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:12,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 08:54:12,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:12,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d374a72214054febe38817 HTTP/1.1" 200 None 2026-04-08 08:54:12,287 - lava_callback - ERROR - Error processing callback for node 69d374a72214054febe38817: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59172 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:54:33,229 - lava_callback - INFO - Received callback for node 69d616854b6493d86166266a INFO: 172.20.0.1:59484 - "POST /node/69d616854b6493d86166266a HTTP/1.0" 202 Accepted 2026-04-08 08:54:33,236 - lava_callback - INFO - Starting async job submit for node 69d616854b6493d86166266a 2026-04-08 08:54:33,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:33,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616854b6493d86166266a HTTP/1.1" 200 None 2026-04-08 08:54:33,277 - lava_callback - DEBUG - Processing node 69d616854b6493d86166266a: ltp-cve 2026-04-08 08:54:33,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:33,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:33,880 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69d616854b6493d86166266a/log.txt.gz 2026-04-08 08:54:34,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:34,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:34,356 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69d616854b6493d86166266a/lava-logs.yaml 2026-04-08 08:54:34,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:34,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:34,521 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69d616854b6493d86166266a/lava_callback.json.gz 2026-04-08 08:54:34,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:34,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616854b6493d86166266a HTTP/1.1" 200 None 2026-04-08 08:54:34,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:34,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:54:34,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:34,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616854b6493d86166266a HTTP/1.1" 200 None 2026-04-08 08:54:34,605 - lava_callback - INFO - Completed processing callback for node 69d616854b6493d86166266a 2026-04-08 08:54:35,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:35,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46208 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:54:52,225 - lava_callback - INFO - Received callback for node 69d374d02214054febe38849 INFO: 172.20.0.1:49824 - "POST /node/69d374d02214054febe38849 HTTP/1.0" 202 Accepted 2026-04-08 08:54:52,234 - lava_callback - INFO - Starting async job submit for node 69d374d02214054febe38849 2026-04-08 08:54:52,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:52,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d374d02214054febe38849 HTTP/1.1" 200 None 2026-04-08 08:54:52,263 - lava_callback - DEBUG - Processing node 69d374d02214054febe38849: baseline-arm64-baylibre 2026-04-08 08:54:52,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d374d02214054febe38849 2026-04-08 08:54:52,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:52,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d374d02214054febe38849 HTTP/1.1" 200 None 2026-04-08 08:54:52,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:52,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 08:54:52,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:52,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d374d02214054febe38849 HTTP/1.1" 200 None 2026-04-08 08:54:52,385 - lava_callback - ERROR - Error processing callback for node 69d374d02214054febe38849: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:54:53,350 - lava_callback - INFO - Received callback for node 69d616f04b6493d8616626eb INFO: 172.20.0.1:49840 - "POST /node/69d616f04b6493d8616626eb HTTP/1.0" 202 Accepted 2026-04-08 08:54:53,359 - lava_callback - INFO - Starting async job submit for node 69d616f04b6493d8616626eb 2026-04-08 08:54:53,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:53,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616f04b6493d8616626eb HTTP/1.1" 200 None 2026-04-08 08:54:53,401 - lava_callback - DEBUG - Processing node 69d616f04b6493d8616626eb: ltp-pty 2026-04-08 08:54:53,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:54,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:54,091 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69d616f04b6493d8616626eb/log.txt.gz 2026-04-08 08:54:54,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:54,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:54,566 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69d616f04b6493d8616626eb/lava-logs.yaml 2026-04-08 08:54:54,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:54:54,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:54:54,746 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69d616f04b6493d8616626eb/lava_callback.json.gz 2026-04-08 08:54:54,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:54,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d616f04b6493d8616626eb HTTP/1.1" 200 None 2026-04-08 08:54:54,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:54,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:54:54,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:54:54,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d616f04b6493d8616626eb HTTP/1.1" 200 None 2026-04-08 08:54:54,851 - lava_callback - INFO - Completed processing callback for node 69d616f04b6493d8616626eb INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:55:05,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:05,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44966 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:55:33,351 - lava_callback - INFO - Received callback for node 69d6170f4b6493d861662722 INFO: 172.20.0.1:48526 - "POST /node/69d6170f4b6493d861662722 HTTP/1.0" 202 Accepted 2026-04-08 08:55:33,357 - lava_callback - INFO - Starting async job submit for node 69d6170f4b6493d861662722 2026-04-08 08:55:33,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:33,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6170f4b6493d861662722 HTTP/1.1" 200 None 2026-04-08 08:55:33,398 - lava_callback - DEBUG - Processing node 69d6170f4b6493d861662722: ltp-smoke 2026-04-08 08:55:33,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:55:33,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:55:33,882 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69d6170f4b6493d861662722/log.txt.gz 2026-04-08 08:55:34,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:55:34,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:55:34,306 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69d6170f4b6493d861662722/lava-logs.yaml 2026-04-08 08:55:34,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:55:34,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:55:34,493 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69d6170f4b6493d861662722/lava_callback.json.gz 2026-04-08 08:55:34,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:34,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6170f4b6493d861662722 HTTP/1.1" 200 None 2026-04-08 08:55:34,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:34,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:55:34,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:34,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6170f4b6493d861662722 HTTP/1.1" 200 None 2026-04-08 08:55:34,591 - lava_callback - INFO - Completed processing callback for node 69d6170f4b6493d861662722 2026-04-08 08:55:35,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:35,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48742 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:55:52,347 - lava_callback - INFO - Received callback for node 69d3750e2214054febe388ad INFO: 172.20.0.1:48880 - "POST /node/69d3750e2214054febe388ad HTTP/1.0" 202 Accepted 2026-04-08 08:55:52,353 - lava_callback - INFO - Starting async job submit for node 69d3750e2214054febe388ad 2026-04-08 08:55:52,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:52,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3750e2214054febe388ad HTTP/1.1" 200 None 2026-04-08 08:55:52,368 - lava_callback - DEBUG - Processing node 69d3750e2214054febe388ad: baseline-arm64-baylibre 2026-04-08 08:55:52,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3750e2214054febe388ad 2026-04-08 08:55:52,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:52,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3750e2214054febe388ad HTTP/1.1" 200 None 2026-04-08 08:55:52,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:52,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 08:55:52,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:55:52,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3750e2214054febe388ad HTTP/1.1" 200 None 2026-04-08 08:55:52,431 - lava_callback - ERROR - Error processing callback for node 69d3750e2214054febe388ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48182 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:56:20,653 - lava_callback - INFO - Received callback for node 69d618254b6493d86166282f INFO: 172.20.0.1:41040 - "POST /node/69d618254b6493d86166282f HTTP/1.0" 202 Accepted 2026-04-08 08:56:20,659 - lava_callback - INFO - Starting async job submit for node 69d618254b6493d86166282f 2026-04-08 08:56:20,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:20,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d618254b6493d86166282f HTTP/1.1" 200 None 2026-04-08 08:56:20,677 - lava_callback - DEBUG - Processing node 69d618254b6493d86166282f: baseline-arm 2026-04-08 08:56:20,682 - lava_callback - WARNING - Failed to upload log 2026-04-08 08:56:20,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:20,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:20,855 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d618254b6493d86166282f/lava-logs.yaml 2026-04-08 08:56:20,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:20,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:20,990 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d618254b6493d86166282f/lava_callback.json.gz 2026-04-08 08:56:20,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:21,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d618254b6493d86166282f HTTP/1.1" 200 None 2026-04-08 08:56:21,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:21,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:56:21,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:21,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d618254b6493d86166282f HTTP/1.1" 200 None 2026-04-08 08:56:21,091 - lava_callback - INFO - Completed processing callback for node 69d618254b6493d86166282f INFO: 172.20.0.1:41052 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:56:35,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:35,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56930 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:56:48,111 - lava_callback - INFO - Received callback for node 69d615d24b6493d8616623e9 INFO: 172.20.0.1:56934 - "POST /node/69d615d24b6493d8616623e9 HTTP/1.0" 202 Accepted 2026-04-08 08:56:48,118 - lava_callback - INFO - Starting async job submit for node 69d615d24b6493d8616623e9 2026-04-08 08:56:48,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:48,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d24b6493d8616623e9 HTTP/1.1" 200 None 2026-04-08 08:56:48,160 - lava_callback - DEBUG - Processing node 69d615d24b6493d8616623e9: baseline-arm64 2026-04-08 08:56:48,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:48,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:49,000 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d24b6493d8616623e9/log.txt.gz 2026-04-08 08:56:49,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:49,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:49,451 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d24b6493d8616623e9/lava-logs.yaml 2026-04-08 08:56:49,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:49,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:49,649 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615d24b6493d8616623e9/lava_callback.json.gz 2026-04-08 08:56:49,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:49,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615d24b6493d8616623e9 HTTP/1.1" 200 None 2026-04-08 08:56:49,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:49,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:56:49,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:49,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615d24b6493d8616623e9 HTTP/1.1" 200 None 2026-04-08 08:56:49,795 - lava_callback - INFO - Completed processing callback for node 69d615d24b6493d8616623e9 2026-04-08 08:56:54,640 - lava_callback - INFO - Received callback for node 69d615e04b6493d861662410 INFO: 172.20.0.1:46580 - "POST /node/69d615e04b6493d861662410 HTTP/1.0" 202 Accepted 2026-04-08 08:56:54,654 - lava_callback - INFO - Starting async job submit for node 69d615e04b6493d861662410 2026-04-08 08:56:54,669 - lava_callback - INFO - Received callback for node 69d6149b4b6493d8616620b9 INFO: 172.20.0.1:46584 - "POST /node/69d6149b4b6493d8616620b9 HTTP/1.0" 202 Accepted 2026-04-08 08:56:54,685 - lava_callback - INFO - Starting async job submit for node 69d6149b4b6493d8616620b9 2026-04-08 08:56:54,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:54,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:54,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149b4b6493d8616620b9 HTTP/1.1" 200 None 2026-04-08 08:56:54,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e04b6493d861662410 HTTP/1.1" 200 None 2026-04-08 08:56:54,812 - lava_callback - DEBUG - Processing node 69d6149b4b6493d8616620b9: baseline-x86 2026-04-08 08:56:54,824 - lava_callback - DEBUG - Processing node 69d615e04b6493d861662410: baseline-nfs-arm64 INFO: 172.20.0.1:46600 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:56:56,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:56,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:56,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:56,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:56,937 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149b4b6493d8616620b9/log.txt.gz 2026-04-08 08:56:56,937 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e04b6493d861662410/log.txt.gz 2026-04-08 08:56:57,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:58,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:58,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:58,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149b4b6493d8616620b9/lava-logs.yaml 2026-04-08 08:56:58,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:58,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:58,356 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e04b6493d861662410/lava-logs.yaml 2026-04-08 08:56:58,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:56:58,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:58,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d6149b4b6493d8616620b9/lava_callback.json.gz 2026-04-08 08:56:58,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:56:58,604 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e04b6493d861662410/lava_callback.json.gz DEBUG: Setup failed for 69d6149b4b6493d8616620b9. Transitting job node result: incomplete -> fail 2026-04-08 08:56:58,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:58,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:58,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6149b4b6493d8616620b9 HTTP/1.1" 200 None 2026-04-08 08:56:58,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:58,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e04b6493d861662410 HTTP/1.1" 200 None 2026-04-08 08:56:58,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:58,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f334b6493d861661fc9 HTTP/1.1" 200 None 2026-04-08 08:56:58,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:58,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:56:58,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:56:58,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615e04b6493d861662410 HTTP/1.1" 200 None 2026-04-08 08:56:58,769 - lava_callback - INFO - Completed processing callback for node 69d615e04b6493d861662410 2026-04-08 08:56:58,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6149b4b6493d8616620b9 HTTP/1.1" 200 None 2026-04-08 08:56:58,942 - lava_callback - INFO - Completed processing callback for node 69d6149b4b6493d8616620b9 2026-04-08 08:57:05,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:05,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60214 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:57:12,366 - lava_callback - INFO - Received callback for node 69d375592214054febe388d1 INFO: 172.20.0.1:51634 - "POST /node/69d375592214054febe388d1 HTTP/1.0" 202 Accepted 2026-04-08 08:57:12,372 - lava_callback - INFO - Starting async job submit for node 69d375592214054febe388d1 2026-04-08 08:57:12,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:12,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d375592214054febe388d1 HTTP/1.1" 200 None 2026-04-08 08:57:12,389 - lava_callback - DEBUG - Processing node 69d375592214054febe388d1: baseline-arm64-baylibre 2026-04-08 08:57:12,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d375592214054febe388d1 2026-04-08 08:57:12,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:12,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d375592214054febe388d1 HTTP/1.1" 200 None 2026-04-08 08:57:12,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:12,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 08:57:12,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:12,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d375592214054febe388d1 HTTP/1.1" 200 None 2026-04-08 08:57:12,441 - lava_callback - ERROR - Error processing callback for node 69d375592214054febe388d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49376 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:57:36,403 - lava_callback - INFO - Received callback for node 69d615cc4b6493d8616623a3 INFO: 172.20.0.1:57878 - "POST /node/69d615cc4b6493d8616623a3 HTTP/1.0" 202 Accepted 2026-04-08 08:57:36,414 - lava_callback - INFO - Starting async job submit for node 69d615cc4b6493d8616623a3 2026-04-08 08:57:36,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:36,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615cc4b6493d8616623a3 HTTP/1.1" 200 None 2026-04-08 08:57:36,468 - lava_callback - DEBUG - Processing node 69d615cc4b6493d8616623a3: baseline-arm64 INFO: 172.20.0.1:57886 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:57:37,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:57:37,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:57:37,206 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cc4b6493d8616623a3/log.txt.gz 2026-04-08 08:57:37,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:57:37,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:57:37,633 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cc4b6493d8616623a3/lava-logs.yaml 2026-04-08 08:57:37,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:57:37,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:57:37,785 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d615cc4b6493d8616623a3/lava_callback.json.gz 2026-04-08 08:57:37,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:37,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615cc4b6493d8616623a3 HTTP/1.1" 200 None 2026-04-08 08:57:37,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:37,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:57:37,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:37,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615cc4b6493d8616623a3 HTTP/1.1" 200 None 2026-04-08 08:57:38,058 - lava_callback - INFO - Completed processing callback for node 69d615cc4b6493d8616623a3 INFO: 172.20.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34996 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:57:52,842 - lava_callback - INFO - Received callback for node 69d615e44b6493d861662419 INFO: 172.20.0.1:42340 - "POST /node/69d615e44b6493d861662419 HTTP/1.0" 202 Accepted 2026-04-08 08:57:52,853 - lava_callback - INFO - Starting async job submit for node 69d615e44b6493d861662419 2026-04-08 08:57:52,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:52,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e44b6493d861662419 HTTP/1.1" 200 None 2026-04-08 08:57:52,908 - lava_callback - DEBUG - Processing node 69d615e44b6493d861662419: baseline-nfs-arm64 2026-04-08 08:57:53,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:57:53,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:57:53,668 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e44b6493d861662419/log.txt.gz 2026-04-08 08:57:53,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:57:54,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:57:54,418 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e44b6493d861662419/lava-logs.yaml 2026-04-08 08:57:54,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:57:54,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:57:54,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615e44b6493d861662419/lava_callback.json.gz 2026-04-08 08:57:54,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:54,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615e44b6493d861662419 HTTP/1.1" 200 None 2026-04-08 08:57:54,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:54,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:57:54,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:57:54,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615e44b6493d861662419 HTTP/1.1" 200 None 2026-04-08 08:57:54,780 - lava_callback - INFO - Completed processing callback for node 69d615e44b6493d861662419 INFO: 172.20.0.1:42354 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:58:05,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:05,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58434 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:58:12,436 - lava_callback - INFO - Received callback for node 69d375a22214054febe38901 INFO: 172.20.0.1:56212 - "POST /node/69d375a22214054febe38901 HTTP/1.0" 202 Accepted 2026-04-08 08:58:12,445 - lava_callback - INFO - Starting async job submit for node 69d375a22214054febe38901 2026-04-08 08:58:12,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:12,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d375a22214054febe38901 HTTP/1.1" 200 None 2026-04-08 08:58:12,490 - lava_callback - DEBUG - Processing node 69d375a22214054febe38901: baseline-arm64-baylibre 2026-04-08 08:58:12,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d375a22214054febe38901 2026-04-08 08:58:12,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:12,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d375a22214054febe38901 HTTP/1.1" 200 None 2026-04-08 08:58:12,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:12,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 08:58:12,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:12,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d375a22214054febe38901 HTTP/1.1" 200 None 2026-04-08 08:58:12,565 - lava_callback - ERROR - Error processing callback for node 69d375a22214054febe38901: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46430 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:58:38,263 - lava_callback - INFO - Received callback for node 69d615df4b6493d86166240d INFO: 172.20.0.1:46440 - "POST /node/69d615df4b6493d86166240d HTTP/1.0" 202 Accepted 2026-04-08 08:58:38,274 - lava_callback - INFO - Starting async job submit for node 69d615df4b6493d86166240d 2026-04-08 08:58:38,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:38,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615df4b6493d86166240d HTTP/1.1" 200 None 2026-04-08 08:58:38,368 - lava_callback - DEBUG - Processing node 69d615df4b6493d86166240d: baseline-nfs-arm64 2026-04-08 08:58:38,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:58:39,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:58:39,103 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615df4b6493d86166240d/log.txt.gz INFO: 172.20.0.1:46452 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:58:39,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:58:39,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:58:39,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615df4b6493d86166240d/lava-logs.yaml 2026-04-08 08:58:39,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:58:39,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:58:39,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615df4b6493d86166240d/lava_callback.json.gz 2026-04-08 08:58:39,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:39,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615df4b6493d86166240d HTTP/1.1" 200 None 2026-04-08 08:58:39,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:39,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 08:58:39,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:58:39,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615df4b6493d86166240d HTTP/1.1" 200 None 2026-04-08 08:58:39,977 - lava_callback - INFO - Completed processing callback for node 69d615df4b6493d86166240d INFO: 172.20.0.1:35074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47712 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:59:05,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:05,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44910 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:59:14,051 - lava_callback - INFO - Received callback for node 69d614a54b6493d86166210c INFO: 172.20.0.1:33364 - "POST /node/69d614a54b6493d86166210c HTTP/1.0" 202 Accepted 2026-04-08 08:59:14,059 - lava_callback - INFO - Starting async job submit for node 69d614a54b6493d86166210c 2026-04-08 08:59:14,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:14,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a54b6493d86166210c HTTP/1.1" 200 None 2026-04-08 08:59:14,157 - lava_callback - DEBUG - Processing node 69d614a54b6493d86166210c: baseline-arm 2026-04-08 08:59:14,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:59:15,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:59:15,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a54b6493d86166210c/log.txt.gz INFO: 172.20.0.1:33370 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 08:59:15,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:59:15,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:59:15,682 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a54b6493d86166210c/lava-logs.yaml 2026-04-08 08:59:15,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 08:59:15,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 08:59:15,860 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d614a54b6493d86166210c/lava_callback.json.gz DEBUG: Setup failed for 69d614a54b6493d86166210c. Transitting job node result: incomplete -> fail 2026-04-08 08:59:16,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:16,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d614a54b6493d86166210c HTTP/1.1" 200 None 2026-04-08 08:59:16,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:16,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 08:59:16,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:16,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d614a54b6493d86166210c HTTP/1.1" 200 None 2026-04-08 08:59:16,220 - lava_callback - INFO - Completed processing callback for node 69d614a54b6493d86166210c INFO: 172.20.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK 2026-04-08 08:59:32,586 - lava_callback - INFO - Received callback for node 69d375e52214054febe3894b INFO: 172.20.0.1:55860 - "POST /node/69d375e52214054febe3894b HTTP/1.0" 202 Accepted 2026-04-08 08:59:32,592 - lava_callback - INFO - Starting async job submit for node 69d375e52214054febe3894b 2026-04-08 08:59:32,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:32,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d375e52214054febe3894b HTTP/1.1" 200 None 2026-04-08 08:59:32,779 - lava_callback - DEBUG - Processing node 69d375e52214054febe3894b: baseline-arm64-baylibre 2026-04-08 08:59:32,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d375e52214054febe3894b 2026-04-08 08:59:32,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:32,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d375e52214054febe3894b HTTP/1.1" 200 None 2026-04-08 08:59:32,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:32,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 08:59:32,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:32,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d375e52214054febe3894b HTTP/1.1" 200 None 2026-04-08 08:59:32,830 - lava_callback - ERROR - Error processing callback for node 69d375e52214054febe3894b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 08:59:35,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 08:59:35,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52370 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:00:32,507 - lava_callback - INFO - Received callback for node 69d3762f2214054febe38974 INFO: 172.20.0.1:46102 - "POST /node/69d3762f2214054febe38974 HTTP/1.0" 202 Accepted 2026-04-08 09:00:32,512 - lava_callback - INFO - Starting async job submit for node 69d3762f2214054febe38974 2026-04-08 09:00:32,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:00:32,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3762f2214054febe38974 HTTP/1.1" 200 None 2026-04-08 09:00:32,528 - lava_callback - DEBUG - Processing node 69d3762f2214054febe38974: baseline-arm64-baylibre 2026-04-08 09:00:32,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3762f2214054febe38974 2026-04-08 09:00:32,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:00:32,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3762f2214054febe38974 HTTP/1.1" 200 None 2026-04-08 09:00:32,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:00:32,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 09:00:32,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:00:32,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3762f2214054febe38974 HTTP/1.1" 200 None 2026-04-08 09:00:32,596 - lava_callback - ERROR - Error processing callback for node 69d3762f2214054febe38974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37444 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:01:32,668 - lava_callback - INFO - Received callback for node 69d376652214054febe389aa INFO: 172.20.0.1:56402 - "POST /node/69d376652214054febe389aa HTTP/1.0" 202 Accepted 2026-04-08 09:01:32,675 - lava_callback - INFO - Starting async job submit for node 69d376652214054febe389aa 2026-04-08 09:01:32,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:01:32,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d376652214054febe389aa HTTP/1.1" 200 None 2026-04-08 09:01:32,691 - lava_callback - DEBUG - Processing node 69d376652214054febe389aa: baseline-arm-baylibre 2026-04-08 09:01:32,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d376652214054febe389aa 2026-04-08 09:01:32,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:01:32,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d376652214054febe389aa HTTP/1.1" 200 None 2026-04-08 09:01:32,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:01:32,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:01:32,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:01:32,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d376652214054febe389aa HTTP/1.1" 200 None 2026-04-08 09:01:32,753 - lava_callback - ERROR - Error processing callback for node 69d376652214054febe389aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55812 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:02:31,243 - lava_callback - INFO - Received callback for node 69d376972214054febe389d8 INFO: 172.20.0.1:34062 - "POST /node/69d376972214054febe389d8 HTTP/1.0" 202 Accepted 2026-04-08 09:02:31,249 - lava_callback - INFO - Starting async job submit for node 69d376972214054febe389d8 2026-04-08 09:02:31,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:02:31,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d376972214054febe389d8 HTTP/1.1" 200 None 2026-04-08 09:02:31,270 - lava_callback - DEBUG - Processing node 69d376972214054febe389d8: baseline-arm64-clabbe 2026-04-08 09:02:31,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d376972214054febe389d8 2026-04-08 09:02:31,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:02:31,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d376972214054febe389d8 HTTP/1.1" 200 None 2026-04-08 09:02:31,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:02:31,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 09:02:31,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:02:31,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d376972214054febe389d8 HTTP/1.1" 200 None 2026-04-08 09:02:31,342 - lava_callback - ERROR - Error processing callback for node 69d376972214054febe389d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38862 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:03:31,227 - lava_callback - INFO - Received callback for node 69d376df2214054febe38a11 INFO: 172.20.0.1:56554 - "POST /node/69d376df2214054febe38a11 HTTP/1.0" 202 Accepted 2026-04-08 09:03:31,232 - lava_callback - INFO - Starting async job submit for node 69d376df2214054febe38a11 2026-04-08 09:03:31,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:03:31,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d376df2214054febe38a11 HTTP/1.1" 200 None 2026-04-08 09:03:31,248 - lava_callback - DEBUG - Processing node 69d376df2214054febe38a11: baseline-arm64-clabbe 2026-04-08 09:03:31,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d376df2214054febe38a11 2026-04-08 09:03:31,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:03:31,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d376df2214054febe38a11 HTTP/1.1" 200 None 2026-04-08 09:03:31,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:03:31,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 09:03:31,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:03:31,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d376df2214054febe38a11 HTTP/1.1" 200 None 2026-04-08 09:03:31,310 - lava_callback - ERROR - Error processing callback for node 69d376df2214054febe38a11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:04:51,284 - lava_callback - INFO - Received callback for node 69d377282214054febe38a38 INFO: 172.20.0.1:51040 - "POST /node/69d377282214054febe38a38 HTTP/1.0" 202 Accepted 2026-04-08 09:04:51,290 - lava_callback - INFO - Starting async job submit for node 69d377282214054febe38a38 2026-04-08 09:04:51,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:04:51,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d377282214054febe38a38 HTTP/1.1" 200 None 2026-04-08 09:04:51,305 - lava_callback - DEBUG - Processing node 69d377282214054febe38a38: baseline-arm64-clabbe 2026-04-08 09:04:51,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d377282214054febe38a38 2026-04-08 09:04:51,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:04:51,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d377282214054febe38a38 HTTP/1.1" 200 None 2026-04-08 09:04:51,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:04:51,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 09:04:51,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:04:51,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d377282214054febe38a38 HTTP/1.1" 200 None 2026-04-08 09:04:51,388 - lava_callback - ERROR - Error processing callback for node 69d377282214054febe38a38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:06:11,430 - lava_callback - INFO - Received callback for node 69d3776e2214054febe38a5c INFO: 172.20.0.1:40378 - "POST /node/69d3776e2214054febe38a5c HTTP/1.0" 202 Accepted 2026-04-08 09:06:11,435 - lava_callback - INFO - Starting async job submit for node 69d3776e2214054febe38a5c 2026-04-08 09:06:11,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:11,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3776e2214054febe38a5c HTTP/1.1" 200 None 2026-04-08 09:06:11,458 - lava_callback - DEBUG - Processing node 69d3776e2214054febe38a5c: baseline-arm64-clabbe 2026-04-08 09:06:11,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3776e2214054febe38a5c 2026-04-08 09:06:11,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:11,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3776e2214054febe38a5c HTTP/1.1" 200 None 2026-04-08 09:06:11,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:11,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 09:06:11,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:11,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3776e2214054febe38a5c HTTP/1.1" 200 None 2026-04-08 09:06:11,519 - lava_callback - ERROR - Error processing callback for node 69d3776e2214054febe38a5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52828 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:06:26,864 - lava_callback - INFO - Received callback for node 69d615dd4b6493d861662406 INFO: 172.20.0.1:52844 - "POST /node/69d615dd4b6493d861662406 HTTP/1.0" 202 Accepted 2026-04-08 09:06:26,877 - lava_callback - INFO - Starting async job submit for node 69d615dd4b6493d861662406 2026-04-08 09:06:26,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615dd4b6493d861662406 HTTP/1.1" 200 None 2026-04-08 09:06:26,948 - lava_callback - DEBUG - Processing node 69d615dd4b6493d861662406: baseline-nfs-arm64 2026-04-08 09:06:28,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:06:28,234 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:06:28,236 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615dd4b6493d861662406/log.txt.gz 2026-04-08 09:06:28,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:06:29,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:06:29,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615dd4b6493d861662406/lava-logs.yaml 2026-04-08 09:06:29,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:06:29,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:06:29,357 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d615dd4b6493d861662406/lava_callback.json.gz DEBUG: Setup failed for 69d615dd4b6493d861662406. Transitting job node result: incomplete -> fail 2026-04-08 09:06:29,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:29,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d615dd4b6493d861662406 HTTP/1.1" 200 None 2026-04-08 09:06:29,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:29,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:06:29,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:29,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d615dd4b6493d861662406 HTTP/1.1" 200 None 2026-04-08 09:06:29,591 - lava_callback - INFO - Completed processing callback for node 69d615dd4b6493d861662406 2026-04-08 09:06:35,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:06:35,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53700 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:07:11,432 - lava_callback - INFO - Received callback for node 69d377b62214054febe38a87 INFO: 172.20.0.1:37898 - "POST /node/69d377b62214054febe38a87 HTTP/1.0" 202 Accepted 2026-04-08 09:07:11,438 - lava_callback - INFO - Starting async job submit for node 69d377b62214054febe38a87 2026-04-08 09:07:11,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:07:11,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d377b62214054febe38a87 HTTP/1.1" 200 None 2026-04-08 09:07:11,474 - lava_callback - DEBUG - Processing node 69d377b62214054febe38a87: baseline-arm64-clabbe 2026-04-08 09:07:11,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d377b62214054febe38a87 2026-04-08 09:07:11,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:07:11,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d377b62214054febe38a87 HTTP/1.1" 200 None 2026-04-08 09:07:11,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:07:11,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 09:07:11,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:07:11,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d377b62214054febe38a87 HTTP/1.1" 200 None 2026-04-08 09:07:11,533 - lava_callback - ERROR - Error processing callback for node 69d377b62214054febe38a87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42512 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:08:31,493 - lava_callback - INFO - Received callback for node 69d377fb2214054febe38ab9 INFO: 172.20.0.1:43562 - "POST /node/69d377fb2214054febe38ab9 HTTP/1.0" 202 Accepted 2026-04-08 09:08:31,499 - lava_callback - INFO - Starting async job submit for node 69d377fb2214054febe38ab9 2026-04-08 09:08:31,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:08:31,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d377fb2214054febe38ab9 HTTP/1.1" 200 None 2026-04-08 09:08:31,515 - lava_callback - DEBUG - Processing node 69d377fb2214054febe38ab9: baseline-arm64-clabbe 2026-04-08 09:08:31,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d377fb2214054febe38ab9 2026-04-08 09:08:31,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:08:31,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d377fb2214054febe38ab9 HTTP/1.1" 200 None 2026-04-08 09:08:31,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:08:31,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0c910691bac272b322da1 HTTP/1.1" 200 None 2026-04-08 09:08:31,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:08:31,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d377fb2214054febe38ab9 HTTP/1.1" 200 None 2026-04-08 09:08:31,572 - lava_callback - ERROR - Error processing callback for node 69d377fb2214054febe38ab9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35644 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:11:50,586 - lava_callback - INFO - Received callback for node 69d61bbb4b6493d861662dcb INFO: 172.20.0.1:46454 - "POST /node/69d61bbb4b6493d861662dcb HTTP/1.0" 202 Accepted 2026-04-08 09:11:50,593 - lava_callback - INFO - Starting async job submit for node 69d61bbb4b6493d861662dcb 2026-04-08 09:11:50,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:11:50,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61bbb4b6493d861662dcb HTTP/1.1" 200 None 2026-04-08 09:11:50,611 - lava_callback - DEBUG - Processing node 69d61bbb4b6493d861662dcb: baseline-nfs-arm64 2026-04-08 09:11:50,612 - lava_callback - WARNING - Failed to upload log 2026-04-08 09:11:50,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:11:50,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:11:50,763 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d61bbb4b6493d861662dcb/lava-logs.yaml 2026-04-08 09:11:50,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:11:50,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:11:50,916 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d61bbb4b6493d861662dcb/lava_callback.json.gz 2026-04-08 09:11:50,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:11:50,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61bbb4b6493d861662dcb HTTP/1.1" 200 None 2026-04-08 09:11:50,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:11:50,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:11:50,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:11:50,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d61bbb4b6493d861662dcb HTTP/1.1" 200 None 2026-04-08 09:11:50,976 - lava_callback - INFO - Completed processing callback for node 69d61bbb4b6493d861662dcb INFO: 172.20.0.1:46466 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:12:05,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:12:05,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44648 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:12:53,523 - lava_callback - INFO - Received callback for node 69d379012214054febe38b57 INFO: 172.20.0.1:39868 - "POST /node/69d379012214054febe38b57 HTTP/1.0" 202 Accepted 2026-04-08 09:12:53,528 - lava_callback - INFO - Starting async job submit for node 69d379012214054febe38b57 2026-04-08 09:12:53,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:12:53,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d379012214054febe38b57 HTTP/1.1" 200 None 2026-04-08 09:12:53,544 - lava_callback - DEBUG - Processing node 69d379012214054febe38b57: baseline-arm-baylibre 2026-04-08 09:12:53,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d379012214054febe38b57 2026-04-08 09:12:53,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:12:53,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d379012214054febe38b57 HTTP/1.1" 200 None 2026-04-08 09:12:53,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:12:53,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:12:53,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:12:53,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d379012214054febe38b57 HTTP/1.1" 200 None 2026-04-08 09:12:53,620 - lava_callback - ERROR - Error processing callback for node 69d379012214054febe38b57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41156 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:14:55,095 - lava_callback - INFO - Received callback for node 69d61c4d4b6493d861662e3d INFO: 172.20.0.1:44642 - "POST /node/69d61c4d4b6493d861662e3d HTTP/1.0" 202 Accepted 2026-04-08 09:14:55,101 - lava_callback - INFO - Starting async job submit for node 69d61c4d4b6493d861662e3d 2026-04-08 09:14:55,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:14:55,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61c4d4b6493d861662e3d HTTP/1.1" 200 None 2026-04-08 09:14:55,142 - lava_callback - DEBUG - Processing node 69d61c4d4b6493d861662e3d: baseline-arm64-broonie 2026-04-08 09:14:55,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:14:55,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:14:55,554 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d61c4d4b6493d861662e3d/log.txt.gz 2026-04-08 09:14:55,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:14:55,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:14:55,890 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d61c4d4b6493d861662e3d/lava-logs.yaml 2026-04-08 09:14:55,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:14:56,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:14:56,078 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d61c4d4b6493d861662e3d/lava_callback.json.gz 2026-04-08 09:14:56,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:14:56,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61c4d4b6493d861662e3d HTTP/1.1" 200 None 2026-04-08 09:14:56,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:14:56,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:14:56,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:14:56,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d61c4d4b6493d861662e3d HTTP/1.1" 200 None 2026-04-08 09:14:56,209 - lava_callback - INFO - Completed processing callback for node 69d61c4d4b6493d861662e3d INFO: 172.20.0.1:44652 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:15:05,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:15:05,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39306 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:15:50,038 - lava_callback - INFO - Received callback for node 69d61c7f4b6493d861662e73 INFO: 172.20.0.1:48640 - "POST /node/69d61c7f4b6493d861662e73 HTTP/1.0" 202 Accepted 2026-04-08 09:15:50,045 - lava_callback - INFO - Starting async job submit for node 69d61c7f4b6493d861662e73 2026-04-08 09:15:50,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:15:50,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61c7f4b6493d861662e73 HTTP/1.1" 200 None 2026-04-08 09:15:50,090 - lava_callback - DEBUG - Processing node 69d61c7f4b6493d861662e73: baseline-arm64 2026-04-08 09:15:50,333 - lava_callback - WARNING - Failed to upload log 2026-04-08 09:15:50,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:15:50,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:15:50,632 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d61c7f4b6493d861662e73/lava-logs.yaml 2026-04-08 09:15:50,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:15:50,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:15:50,743 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d61c7f4b6493d861662e73/lava_callback.json.gz 2026-04-08 09:15:50,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:15:50,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61c7f4b6493d861662e73 HTTP/1.1" 200 None 2026-04-08 09:15:50,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:15:50,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:15:50,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:15:50,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d61c7f4b6493d861662e73 HTTP/1.1" 200 None 2026-04-08 09:15:50,881 - lava_callback - INFO - Completed processing callback for node 69d61c7f4b6493d861662e73 INFO: 172.20.0.1:48648 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:16:05,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:16:05,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36728 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:17:33,998 - lava_callback - INFO - Received callback for node 69d61cd94b6493d861662ea5 INFO: 172.20.0.1:45820 - "POST /node/69d61cd94b6493d861662ea5 HTTP/1.0" 202 Accepted 2026-04-08 09:17:34,005 - lava_callback - INFO - Starting async job submit for node 69d61cd94b6493d861662ea5 2026-04-08 09:17:34,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:34,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61cd94b6493d861662ea5 HTTP/1.1" 200 None 2026-04-08 09:17:34,045 - lava_callback - DEBUG - Processing node 69d61cd94b6493d861662ea5: baseline-arm64-broonie 2026-04-08 09:17:34,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:17:34,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:17:34,465 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d61cd94b6493d861662ea5/log.txt.gz 2026-04-08 09:17:34,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:17:34,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:17:34,824 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d61cd94b6493d861662ea5/lava-logs.yaml 2026-04-08 09:17:34,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:17:35,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:17:35,007 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d61cd94b6493d861662ea5/lava_callback.json.gz 2026-04-08 09:17:35,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:35,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61cd94b6493d861662ea5 HTTP/1.1" 200 None 2026-04-08 09:17:35,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:35,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:17:35,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:35,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d61cd94b6493d861662ea5 HTTP/1.1" 200 None 2026-04-08 09:17:35,115 - lava_callback - INFO - Completed processing callback for node 69d61cd94b6493d861662ea5 2026-04-08 09:17:35,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:35,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46798 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:17:50,214 - lava_callback - INFO - Received callback for node 69d61d0c4b6493d861662eb4 INFO: 172.20.0.1:54054 - "POST /node/69d61d0c4b6493d861662eb4 HTTP/1.0" 202 Accepted 2026-04-08 09:17:50,263 - lava_callback - INFO - Starting async job submit for node 69d61d0c4b6493d861662eb4 2026-04-08 09:17:50,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:50,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61d0c4b6493d861662eb4 HTTP/1.1" 200 None 2026-04-08 09:17:50,308 - lava_callback - DEBUG - Processing node 69d61d0c4b6493d861662eb4: baseline-arm64 2026-04-08 09:17:50,430 - lava_callback - WARNING - Failed to upload log 2026-04-08 09:17:50,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:17:50,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:17:50,673 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d61d0c4b6493d861662eb4/lava-logs.yaml 2026-04-08 09:17:50,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:17:50,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:17:50,793 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d61d0c4b6493d861662eb4/lava_callback.json.gz 2026-04-08 09:17:50,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:50,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61d0c4b6493d861662eb4 HTTP/1.1" 200 None 2026-04-08 09:17:50,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:50,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:17:50,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:17:50,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d61d0c4b6493d861662eb4 HTTP/1.1" 200 None 2026-04-08 09:17:50,955 - lava_callback - INFO - Completed processing callback for node 69d61d0c4b6493d861662eb4 INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:18:05,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:18:05,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60070 - "GET /node/694729483daac48063598230%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60076 - "GET /node/699040db985e4efca431ed6a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51680 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:25:12,243 - lava_callback - INFO - Received callback for node 69d37bf32214054febe38d96 INFO: 172.20.0.1:51048 - "POST /node/69d37bf32214054febe38d96 HTTP/1.0" 202 Accepted 2026-04-08 09:25:12,250 - lava_callback - INFO - Starting async job submit for node 69d37bf32214054febe38d96 2026-04-08 09:25:12,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:25:12,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37bf32214054febe38d96 HTTP/1.1" 200 None 2026-04-08 09:25:12,268 - lava_callback - DEBUG - Processing node 69d37bf32214054febe38d96: baseline-arm-clabbe 2026-04-08 09:25:12,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37bf32214054febe38d96 2026-04-08 09:25:12,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:25:12,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37bf32214054febe38d96 HTTP/1.1" 200 None 2026-04-08 09:25:12,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:25:12,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:25:12,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:25:12,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37bf32214054febe38d96 HTTP/1.1" 200 None 2026-04-08 09:25:12,324 - lava_callback - ERROR - Error processing callback for node 69d37bf32214054febe38d96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36740 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:27:19,091 - lava_callback - INFO - Received callback for node 69d61eff4b6493d8616630b5 INFO: 172.20.0.1:36742 - "POST /node/69d61eff4b6493d8616630b5 HTTP/1.0" 202 Accepted 2026-04-08 09:27:19,098 - lava_callback - INFO - Starting async job submit for node 69d61eff4b6493d8616630b5 2026-04-08 09:27:19,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:27:19,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61eff4b6493d8616630b5 HTTP/1.1" 200 None 2026-04-08 09:27:19,165 - lava_callback - DEBUG - Processing node 69d61eff4b6493d8616630b5: baseline-arm64 2026-04-08 09:27:19,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:27:19,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:27:19,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d61eff4b6493d8616630b5/log.txt.gz 2026-04-08 09:27:19,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:27:20,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:27:20,037 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d61eff4b6493d8616630b5/lava-logs.yaml 2026-04-08 09:27:20,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:27:20,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:27:20,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d61eff4b6493d8616630b5/lava_callback.json.gz 2026-04-08 09:27:20,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:27:20,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d61eff4b6493d8616630b5 HTTP/1.1" 200 None 2026-04-08 09:27:20,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:27:20,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:27:20,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:27:20,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d61eff4b6493d8616630b5 HTTP/1.1" 200 None 2026-04-08 09:27:20,351 - lava_callback - INFO - Completed processing callback for node 69d61eff4b6493d8616630b5 INFO: 172.20.0.1:56604 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:27:35,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:27:35,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56450 - "GET /node/6987f1dae31b58f834f1dd13%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56466 - "GET /node/6992ea8431166481c8dd8da0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56468 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:28:52,376 - lava_callback - INFO - Received callback for node 69d37cc02214054febe38e48 INFO: 172.20.0.1:58288 - "POST /node/69d37cc02214054febe38e48 HTTP/1.0" 202 Accepted 2026-04-08 09:28:52,381 - lava_callback - INFO - Starting async job submit for node 69d37cc02214054febe38e48 2026-04-08 09:28:52,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:28:52,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37cc02214054febe38e48 HTTP/1.1" 200 None 2026-04-08 09:28:52,407 - lava_callback - DEBUG - Processing node 69d37cc02214054febe38e48: baseline-arm-clabbe 2026-04-08 09:28:52,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37cc02214054febe38e48 2026-04-08 09:28:52,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:28:52,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37cc02214054febe38e48 HTTP/1.1" 200 None 2026-04-08 09:28:52,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:28:52,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:28:52,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:28:52,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37cc02214054febe38e48 HTTP/1.1" 200 None 2026-04-08 09:28:52,461 - lava_callback - ERROR - Error processing callback for node 69d37cc02214054febe38e48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47048 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:32:12,535 - lava_callback - INFO - Received callback for node 69d37d932214054febe38f0c INFO: 172.20.0.1:50226 - "POST /node/69d37d932214054febe38f0c HTTP/1.0" 202 Accepted 2026-04-08 09:32:12,541 - lava_callback - INFO - Starting async job submit for node 69d37d932214054febe38f0c 2026-04-08 09:32:12,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:32:12,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37d932214054febe38f0c HTTP/1.1" 200 None 2026-04-08 09:32:12,569 - lava_callback - DEBUG - Processing node 69d37d932214054febe38f0c: baseline-arm-clabbe 2026-04-08 09:32:12,569 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37d932214054febe38f0c 2026-04-08 09:32:12,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:32:12,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37d932214054febe38f0c HTTP/1.1" 200 None 2026-04-08 09:32:12,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:32:12,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:32:12,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:32:12,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37d932214054febe38f0c HTTP/1.1" 200 None 2026-04-08 09:32:12,648 - lava_callback - ERROR - Error processing callback for node 69d37d932214054febe38f0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34664 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:33:12,592 - lava_callback - INFO - Received callback for node 69d37dd52214054febe38f47 INFO: 172.20.0.1:38666 - "POST /node/69d37dd52214054febe38f47 HTTP/1.0" 202 Accepted 2026-04-08 09:33:12,597 - lava_callback - INFO - Starting async job submit for node 69d37dd52214054febe38f47 2026-04-08 09:33:12,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:12,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37dd52214054febe38f47 HTTP/1.1" 200 None 2026-04-08 09:33:12,614 - lava_callback - DEBUG - Processing node 69d37dd52214054febe38f47: baseline-arm-clabbe 2026-04-08 09:33:12,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37dd52214054febe38f47 2026-04-08 09:33:12,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:12,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37dd52214054febe38f47 HTTP/1.1" 200 None 2026-04-08 09:33:12,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:12,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:33:12,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:12,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37dd52214054febe38f47 HTTP/1.1" 200 None 2026-04-08 09:33:12,685 - lava_callback - ERROR - Error processing callback for node 69d37dd52214054febe38f47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38676 - "GET /node/698e2a84952a14bf500e4122%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38692 - "GET /node/698f5ee83013abef95cf9750%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:33:52,581 - lava_callback - INFO - Received callback for node 69d37df62214054febe38f6a INFO: 172.20.0.1:33130 - "POST /node/69d37df62214054febe38f6a HTTP/1.0" 202 Accepted 2026-04-08 09:33:52,587 - lava_callback - INFO - Starting async job submit for node 69d37df62214054febe38f6a 2026-04-08 09:33:52,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:52,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37df62214054febe38f6a HTTP/1.1" 200 None 2026-04-08 09:33:52,605 - lava_callback - DEBUG - Processing node 69d37df62214054febe38f6a: baseline-arm-clabbe 2026-04-08 09:33:52,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37df62214054febe38f6a 2026-04-08 09:33:52,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:52,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37df62214054febe38f6a HTTP/1.1" 200 None 2026-04-08 09:33:52,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:52,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:33:52,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:33:52,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37df62214054febe38f6a HTTP/1.1" 200 None 2026-04-08 09:33:52,711 - lava_callback - ERROR - Error processing callback for node 69d37df62214054febe38f6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49922 - "GET /node/69938434cebe99dd8e06f887%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43876 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:35:32,680 - lava_callback - INFO - Received callback for node 69d37e5b2214054febe38f97 INFO: 172.20.0.1:36462 - "POST /node/69d37e5b2214054febe38f97 HTTP/1.0" 202 Accepted 2026-04-08 09:35:32,686 - lava_callback - INFO - Starting async job submit for node 69d37e5b2214054febe38f97 2026-04-08 09:35:32,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:35:32,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37e5b2214054febe38f97 HTTP/1.1" 200 None 2026-04-08 09:35:32,702 - lava_callback - DEBUG - Processing node 69d37e5b2214054febe38f97: baseline-arm-clabbe 2026-04-08 09:35:32,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37e5b2214054febe38f97 2026-04-08 09:35:32,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:35:32,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37e5b2214054febe38f97 HTTP/1.1" 200 None 2026-04-08 09:35:32,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:35:32,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2669cb08ea23a1f6806e HTTP/1.1" 200 None 2026-04-08 09:35:32,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:35:32,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37e5b2214054febe38f97 HTTP/1.1" 200 None 2026-04-08 09:35:32,770 - lava_callback - ERROR - Error processing callback for node 69d37e5b2214054febe38f97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37162 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:36:14,560 - lava_callback - INFO - Received callback for node 69d37e7c2214054febe38fa7 INFO: 172.20.0.1:42612 - "POST /node/69d37e7c2214054febe38fa7 HTTP/1.0" 202 Accepted 2026-04-08 09:36:14,566 - lava_callback - INFO - Starting async job submit for node 69d37e7c2214054febe38fa7 2026-04-08 09:36:14,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:36:14,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37e7c2214054febe38fa7 HTTP/1.1" 200 None 2026-04-08 09:36:14,582 - lava_callback - DEBUG - Processing node 69d37e7c2214054febe38fa7: baseline-arm-baylibre 2026-04-08 09:36:14,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37e7c2214054febe38fa7 2026-04-08 09:36:14,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:36:14,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37e7c2214054febe38fa7 HTTP/1.1" 200 None 2026-04-08 09:36:14,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:36:14,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:36:14,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:36:14,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37e7c2214054febe38fa7 HTTP/1.1" 200 None 2026-04-08 09:36:14,635 - lava_callback - ERROR - Error processing callback for node 69d37e7c2214054febe38fa7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41108 - "GET /node/6985d08571989fb141499b85%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41122 - "GET /node/693b7f2141fb4c950976421c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37836 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:38:54,683 - lava_callback - INFO - Received callback for node 69d37f252214054febe39047 INFO: 172.20.0.1:36454 - "POST /node/69d37f252214054febe39047 HTTP/1.0" 202 Accepted 2026-04-08 09:38:54,689 - lava_callback - INFO - Starting async job submit for node 69d37f252214054febe39047 2026-04-08 09:38:54,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:54,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37f252214054febe39047 HTTP/1.1" 200 None 2026-04-08 09:38:54,705 - lava_callback - DEBUG - Processing node 69d37f252214054febe39047: baseline-arm-baylibre 2026-04-08 09:38:54,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37f252214054febe39047 2026-04-08 09:38:54,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:54,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37f252214054febe39047 HTTP/1.1" 200 None 2026-04-08 09:38:54,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:54,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:38:54,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:54,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37f252214054febe39047 HTTP/1.1" 200 None 2026-04-08 09:38:54,767 - lava_callback - ERROR - Error processing callback for node 69d37f252214054febe39047: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 09:38:54,945 - lava_callback - INFO - Received callback for node 69d6213f4b6493d86166323a INFO: 172.20.0.1:36470 - "POST /node/69d6213f4b6493d86166323a HTTP/1.0" 202 Accepted 2026-04-08 09:38:54,952 - lava_callback - INFO - Starting async job submit for node 69d6213f4b6493d86166323a 2026-04-08 09:38:54,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:54,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6213f4b6493d86166323a HTTP/1.1" 200 None 2026-04-08 09:38:54,996 - lava_callback - DEBUG - Processing node 69d6213f4b6493d86166323a: ltp-cap_bounds 2026-04-08 09:38:55,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:38:55,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:38:55,588 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69d6213f4b6493d86166323a/log.txt.gz 2026-04-08 09:38:55,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:38:56,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:38:56,021 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69d6213f4b6493d86166323a/lava-logs.yaml 2026-04-08 09:38:56,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:38:56,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:38:56,202 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69d6213f4b6493d86166323a/lava_callback.json.gz 2026-04-08 09:38:56,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:56,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6213f4b6493d86166323a HTTP/1.1" 200 None 2026-04-08 09:38:56,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:56,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 09:38:56,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:38:56,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6213f4b6493d86166323a HTTP/1.1" 200 None 2026-04-08 09:38:56,307 - lava_callback - INFO - Completed processing callback for node 69d6213f4b6493d86166323a INFO: 172.20.0.1:36484 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:39:05,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:39:05,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51654 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 09:39:16,120 - lava_callback - INFO - Received callback for node 69d621604b6493d861663241 INFO: 172.20.0.1:51668 - "POST /node/69d621604b6493d861663241 HTTP/1.0" 202 Accepted 2026-04-08 09:39:16,128 - lava_callback - INFO - Starting async job submit for node 69d621604b6493d861663241 2026-04-08 09:39:16,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:39:16,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d621604b6493d861663241 HTTP/1.1" 200 None 2026-04-08 09:39:16,171 - lava_callback - DEBUG - Processing node 69d621604b6493d861663241: ltp-controllers INFO: 172.20.0.1:51672 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:39:16,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:39:16,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:39:16,900 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69d621604b6493d861663241/log.txt.gz 2026-04-08 09:39:17,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:39:17,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:39:17,399 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69d621604b6493d861663241/lava-logs.yaml 2026-04-08 09:39:17,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:39:17,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:39:17,577 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69d621604b6493d861663241/lava_callback.json.gz 2026-04-08 09:39:17,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:39:17,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d621604b6493d861663241 HTTP/1.1" 200 None 2026-04-08 09:39:17,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:39:17,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 09:39:17,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:39:17,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d621604b6493d861663241 HTTP/1.1" 200 None 2026-04-08 09:39:17,680 - lava_callback - INFO - Completed processing callback for node 69d621604b6493d861663241 INFO: 172.20.0.1:54542 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:39:35,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:39:35,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59172 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:41:54,826 - lava_callback - INFO - Received callback for node 69d37fd22214054febe390eb INFO: 172.20.0.1:58718 - "POST /node/69d37fd22214054febe390eb HTTP/1.0" 202 Accepted 2026-04-08 09:41:54,832 - lava_callback - INFO - Starting async job submit for node 69d37fd22214054febe390eb 2026-04-08 09:41:54,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:41:54,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37fd22214054febe390eb HTTP/1.1" 200 None 2026-04-08 09:41:54,850 - lava_callback - DEBUG - Processing node 69d37fd22214054febe390eb: baseline-arm-baylibre 2026-04-08 09:41:54,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37fd22214054febe390eb 2026-04-08 09:41:54,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:41:54,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37fd22214054febe390eb HTTP/1.1" 200 None 2026-04-08 09:41:54,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:41:54,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:41:54,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:41:54,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37fd22214054febe390eb HTTP/1.1" 200 None 2026-04-08 09:41:54,926 - lava_callback - ERROR - Error processing callback for node 69d37fd22214054febe390eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46226 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:42:33,011 - lava_callback - INFO - Received callback for node 69d37ff72214054febe390ff INFO: 172.20.0.1:33910 - "POST /node/69d37ff72214054febe390ff HTTP/1.0" 202 Accepted 2026-04-08 09:42:33,017 - lava_callback - INFO - Starting async job submit for node 69d37ff72214054febe390ff 2026-04-08 09:42:33,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:42:33,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37ff72214054febe390ff HTTP/1.1" 200 None 2026-04-08 09:42:33,033 - lava_callback - DEBUG - Processing node 69d37ff72214054febe390ff: baseline-arm-clabbe 2026-04-08 09:42:33,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d37ff72214054febe390ff 2026-04-08 09:42:33,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:42:33,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d37ff72214054febe390ff HTTP/1.1" 200 None 2026-04-08 09:42:33,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:42:33,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:42:33,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:42:33,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d37ff72214054febe390ff HTTP/1.1" 200 None 2026-04-08 09:42:33,107 - lava_callback - ERROR - Error processing callback for node 69d37ff72214054febe390ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44576 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:43:12,964 - lava_callback - INFO - Received callback for node 69d3801b2214054febe39116 INFO: 172.20.0.1:53860 - "POST /node/69d3801b2214054febe39116 HTTP/1.0" 202 Accepted 2026-04-08 09:43:12,970 - lava_callback - INFO - Starting async job submit for node 69d3801b2214054febe39116 2026-04-08 09:43:12,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:43:12,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3801b2214054febe39116 HTTP/1.1" 200 None 2026-04-08 09:43:12,986 - lava_callback - DEBUG - Processing node 69d3801b2214054febe39116: baseline-arm-clabbe 2026-04-08 09:43:12,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3801b2214054febe39116 2026-04-08 09:43:12,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:43:13,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3801b2214054febe39116 HTTP/1.1" 200 None 2026-04-08 09:43:13,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:43:13,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:43:13,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:43:13,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3801b2214054febe39116 HTTP/1.1" 200 None 2026-04-08 09:43:13,049 - lava_callback - ERROR - Error processing callback for node 69d3801b2214054febe39116: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET /node/6952cf479912bd6bbcea4012%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41316 - "GET /node/69845ff63a8748f2820863c2%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-08 09:44:13,097 - lava_callback - INFO - Received callback for node 69d3805e2214054febe39155 INFO: 172.20.0.1:41320 - "POST /node/69d3805e2214054febe39155 HTTP/1.0" 202 Accepted 2026-04-08 09:44:13,104 - lava_callback - INFO - Starting async job submit for node 69d3805e2214054febe39155 2026-04-08 09:44:13,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:13,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3805e2214054febe39155 HTTP/1.1" 200 None 2026-04-08 09:44:13,120 - lava_callback - DEBUG - Processing node 69d3805e2214054febe39155: baseline-arm-clabbe 2026-04-08 09:44:13,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3805e2214054febe39155 2026-04-08 09:44:13,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:13,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3805e2214054febe39155 HTTP/1.1" 200 None 2026-04-08 09:44:13,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:13,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:44:13,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:13,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3805e2214054febe39155 HTTP/1.1" 200 None 2026-04-08 09:44:13,182 - lava_callback - ERROR - Error processing callback for node 69d3805e2214054febe39155: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:44:53,048 - lava_callback - INFO - Received callback for node 69d3807e2214054febe39177 INFO: 172.20.0.1:39814 - "POST /node/69d3807e2214054febe39177 HTTP/1.0" 202 Accepted 2026-04-08 09:44:53,055 - lava_callback - INFO - Starting async job submit for node 69d3807e2214054febe39177 2026-04-08 09:44:53,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:53,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3807e2214054febe39177 HTTP/1.1" 200 None 2026-04-08 09:44:53,077 - lava_callback - DEBUG - Processing node 69d3807e2214054febe39177: baseline-arm-clabbe 2026-04-08 09:44:53,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3807e2214054febe39177 2026-04-08 09:44:53,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:53,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3807e2214054febe39177 HTTP/1.1" 200 None 2026-04-08 09:44:53,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:53,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:44:53,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:53,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3807e2214054febe39177 HTTP/1.1" 200 None 2026-04-08 09:44:53,141 - lava_callback - ERROR - Error processing callback for node 69d3807e2214054febe39177: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-08 09:44:54,980 - lava_callback - INFO - Received callback for node 69d623774b6493d861663403 INFO: 172.20.0.1:39816 - "POST /node/69d623774b6493d861663403 HTTP/1.0" 202 Accepted 2026-04-08 09:44:54,986 - lava_callback - INFO - Starting async job submit for node 69d623774b6493d861663403 2026-04-08 09:44:54,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:55,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d623774b6493d861663403 HTTP/1.1" 200 None 2026-04-08 09:44:55,005 - lava_callback - DEBUG - Processing node 69d623774b6493d861663403: baseline-arm 2026-04-08 09:44:55,011 - lava_callback - WARNING - Failed to upload log 2026-04-08 09:44:55,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:44:55,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:44:55,172 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d623774b6493d861663403/lava-logs.yaml 2026-04-08 09:44:55,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:44:55,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:44:55,343 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d623774b6493d861663403/lava_callback.json.gz 2026-04-08 09:44:55,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:55,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d623774b6493d861663403 HTTP/1.1" 200 None 2026-04-08 09:44:55,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:55,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 09:44:55,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:44:55,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d623774b6493d861663403 HTTP/1.1" 200 None 2026-04-08 09:44:55,450 - lava_callback - INFO - Completed processing callback for node 69d623774b6493d861663403 INFO: 172.20.0.1:39826 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:45:05,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:05,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33148 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:45:13,046 - lava_callback - INFO - Received callback for node 69d3809e2214054febe391a0 INFO: 172.20.0.1:40536 - "POST /node/69d3809e2214054febe391a0 HTTP/1.0" 202 Accepted 2026-04-08 09:45:13,055 - lava_callback - INFO - Starting async job submit for node 69d3809e2214054febe391a0 2026-04-08 09:45:13,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:13,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3809e2214054febe391a0 HTTP/1.1" 200 None 2026-04-08 09:45:13,076 - lava_callback - DEBUG - Processing node 69d3809e2214054febe391a0: baseline-arm-clabbe 2026-04-08 09:45:13,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3809e2214054febe391a0 2026-04-08 09:45:13,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:13,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3809e2214054febe391a0 HTTP/1.1" 200 None 2026-04-08 09:45:13,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:13,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-08 09:45:13,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:13,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3809e2214054febe391a0 HTTP/1.1" 200 None 2026-04-08 09:45:13,147 - lava_callback - ERROR - Error processing callback for node 69d3809e2214054febe391a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40550 - "GET /node/69101abdf4a91d81a0ec2321%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56612 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:45:30,030 - lava_callback - INFO - Received callback for node 69d6226f4b6493d86166333c INFO: 172.20.0.1:54700 - "POST /node/69d6226f4b6493d86166333c HTTP/1.0" 202 Accepted 2026-04-08 09:45:30,038 - lava_callback - INFO - Starting async job submit for node 69d6226f4b6493d86166333c 2026-04-08 09:45:30,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:30,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6226f4b6493d86166333c HTTP/1.1" 200 None 2026-04-08 09:45:30,089 - lava_callback - DEBUG - Processing node 69d6226f4b6493d86166333c: baseline-arm64 2026-04-08 09:45:30,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:45:30,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:45:30,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6226f4b6493d86166333c/log.txt.gz 2026-04-08 09:45:30,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:45:31,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:45:31,015 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6226f4b6493d86166333c/lava-logs.yaml 2026-04-08 09:45:31,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:45:31,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:45:31,181 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6226f4b6493d86166333c/lava_callback.json.gz DEBUG: Setup failed for 69d6226f4b6493d86166333c. Transitting job node result: incomplete -> fail 2026-04-08 09:45:31,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:31,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6226f4b6493d86166333c HTTP/1.1" 200 None 2026-04-08 09:45:31,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:31,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:45:31,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:31,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6226f4b6493d86166333c HTTP/1.1" 200 None 2026-04-08 09:45:31,406 - lava_callback - INFO - Completed processing callback for node 69d6226f4b6493d86166333c 2026-04-08 09:45:35,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:45:35,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55570 - "GET /node/698401cf969566dcb0d2b435%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42666 - "GET /node/694a21282060d3d8479208e8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42682 - "GET /node/698b34403b3beb19e761a1e5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60120 - "GET / HTTP/1.0" 200 OK 2026-04-08 09:49:29,633 - lava_callback - INFO - Received callback for node 69d624674b6493d86166347c INFO: 172.20.0.1:60134 - "POST /node/69d624674b6493d86166347c HTTP/1.0" 202 Accepted 2026-04-08 09:49:29,643 - lava_callback - INFO - Starting async job submit for node 69d624674b6493d86166347c 2026-04-08 09:49:29,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:49:29,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d624674b6493d86166347c HTTP/1.1" 200 None 2026-04-08 09:49:29,718 - lava_callback - DEBUG - Processing node 69d624674b6493d86166347c: baseline-arm64 2026-04-08 09:49:30,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:49:30,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:49:30,266 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d624674b6493d86166347c/log.txt.gz 2026-04-08 09:49:30,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:49:30,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:49:30,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d624674b6493d86166347c/lava-logs.yaml 2026-04-08 09:49:30,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 09:49:30,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 09:49:30,900 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d624674b6493d86166347c/lava_callback.json.gz 2026-04-08 09:49:30,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:49:30,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d624674b6493d86166347c HTTP/1.1" 200 None 2026-04-08 09:49:30,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:49:30,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 09:49:30,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:49:31,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d624674b6493d86166347c HTTP/1.1" 200 None 2026-04-08 09:49:31,065 - lava_callback - INFO - Completed processing callback for node 69d624674b6493d86166347c 2026-04-08 09:49:35,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 09:49:35,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41720 - "GET /node/698f59243013abef95cf7221%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41736 - "GET /node/6985d74a71989fb14149a759%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53000 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:03:01,219 - lava_callback - INFO - Received callback for node 69d6241b4b6493d861663456 INFO: 172.20.0.1:40442 - "POST /node/69d6241b4b6493d861663456 HTTP/1.0" 202 Accepted 2026-04-08 10:03:01,226 - lava_callback - INFO - Starting async job submit for node 69d6241b4b6493d861663456 2026-04-08 10:03:01,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:01,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6241b4b6493d861663456 HTTP/1.1" 200 None 2026-04-08 10:03:01,288 - lava_callback - DEBUG - Processing node 69d6241b4b6493d861663456: baseline-x86 2026-04-08 10:03:01,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:03:02,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:03:02,183 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d6241b4b6493d861663456/log.txt.gz 2026-04-08 10:03:02,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:03:02,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:03:02,861 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d6241b4b6493d861663456/lava-logs.yaml 2026-04-08 10:03:02,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:03:03,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:03:03,073 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d6241b4b6493d861663456/lava_callback.json.gz DEBUG: Setup failed for 69d6241b4b6493d861663456. Transitting job node result: incomplete -> fail 2026-04-08 10:03:03,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:03,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6241b4b6493d861663456 HTTP/1.1" 200 None 2026-04-08 10:03:03,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:03,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f334b6493d861661fc9 HTTP/1.1" 200 None 2026-04-08 10:03:03,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:03,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6241b4b6493d861663456 HTTP/1.1" 200 None 2026-04-08 10:03:03,351 - lava_callback - INFO - Completed processing callback for node 69d6241b4b6493d861663456 2026-04-08 10:03:05,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:05,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35408 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:03:56,205 - lava_callback - INFO - Received callback for node 69d384fd2214054febe39515 INFO: 172.20.0.1:46514 - "POST /node/69d384fd2214054febe39515 HTTP/1.0" 202 Accepted 2026-04-08 10:03:56,210 - lava_callback - INFO - Starting async job submit for node 69d384fd2214054febe39515 2026-04-08 10:03:56,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:56,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d384fd2214054febe39515 HTTP/1.1" 200 None 2026-04-08 10:03:56,259 - lava_callback - DEBUG - Processing node 69d384fd2214054febe39515: baseline-arm64-baylibre 2026-04-08 10:03:56,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d384fd2214054febe39515 2026-04-08 10:03:56,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:56,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d384fd2214054febe39515 HTTP/1.1" 200 None 2026-04-08 10:03:56,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:56,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:03:56,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:03:56,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d384fd2214054febe39515 HTTP/1.1" 200 None 2026-04-08 10:03:56,314 - lava_callback - ERROR - Error processing callback for node 69d384fd2214054febe39515: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50868 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:04:56,037 - lava_callback - INFO - Received callback for node 69d385432214054febe3953a INFO: 172.20.0.1:42138 - "POST /node/69d385432214054febe3953a HTTP/1.0" 202 Accepted 2026-04-08 10:04:56,043 - lava_callback - INFO - Starting async job submit for node 69d385432214054febe3953a 2026-04-08 10:04:56,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:04:56,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d385432214054febe3953a HTTP/1.1" 200 None 2026-04-08 10:04:56,061 - lava_callback - DEBUG - Processing node 69d385432214054febe3953a: baseline-arm64-baylibre 2026-04-08 10:04:56,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d385432214054febe3953a 2026-04-08 10:04:56,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:04:56,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d385432214054febe3953a HTTP/1.1" 200 None 2026-04-08 10:04:56,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:04:56,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:04:56,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:04:56,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d385432214054febe3953a HTTP/1.1" 200 None 2026-04-08 10:04:56,136 - lava_callback - ERROR - Error processing callback for node 69d385432214054febe3953a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37682 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:06:16,266 - lava_callback - INFO - Received callback for node 69d385872214054febe39581 INFO: 172.20.0.1:37698 - "POST /node/69d385872214054febe39581 HTTP/1.0" 202 Accepted 2026-04-08 10:06:16,273 - lava_callback - INFO - Starting async job submit for node 69d385872214054febe39581 2026-04-08 10:06:16,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:06:16,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d385872214054febe39581 HTTP/1.1" 200 None 2026-04-08 10:06:16,290 - lava_callback - DEBUG - Processing node 69d385872214054febe39581: baseline-arm64-baylibre 2026-04-08 10:06:16,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d385872214054febe39581 2026-04-08 10:06:16,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:06:16,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d385872214054febe39581 HTTP/1.1" 200 None 2026-04-08 10:06:16,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:06:16,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:06:16,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:06:16,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d385872214054febe39581 HTTP/1.1" 200 None 2026-04-08 10:06:16,343 - lava_callback - ERROR - Error processing callback for node 69d385872214054febe39581: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:07:16,202 - lava_callback - INFO - Received callback for node 69d385cf2214054febe395ab INFO: 172.20.0.1:57638 - "POST /node/69d385cf2214054febe395ab HTTP/1.0" 202 Accepted 2026-04-08 10:07:16,208 - lava_callback - INFO - Starting async job submit for node 69d385cf2214054febe395ab 2026-04-08 10:07:16,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:07:16,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d385cf2214054febe395ab HTTP/1.1" 200 None 2026-04-08 10:07:16,224 - lava_callback - DEBUG - Processing node 69d385cf2214054febe395ab: baseline-arm64-baylibre 2026-04-08 10:07:16,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d385cf2214054febe395ab 2026-04-08 10:07:16,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:07:16,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d385cf2214054febe395ab HTTP/1.1" 200 None 2026-04-08 10:07:16,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:07:16,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:07:16,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:07:16,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d385cf2214054febe395ab HTTP/1.1" 200 None 2026-04-08 10:07:16,288 - lava_callback - ERROR - Error processing callback for node 69d385cf2214054febe395ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55688 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:08:36,289 - lava_callback - INFO - Received callback for node 69d386142214054febe395e2 INFO: 172.20.0.1:60466 - "POST /node/69d386142214054febe395e2 HTTP/1.0" 202 Accepted 2026-04-08 10:08:36,295 - lava_callback - INFO - Starting async job submit for node 69d386142214054febe395e2 2026-04-08 10:08:36,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:08:36,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d386142214054febe395e2 HTTP/1.1" 200 None 2026-04-08 10:08:36,311 - lava_callback - DEBUG - Processing node 69d386142214054febe395e2: baseline-arm64-baylibre 2026-04-08 10:08:36,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d386142214054febe395e2 2026-04-08 10:08:36,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:08:36,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d386142214054febe395e2 HTTP/1.1" 200 None 2026-04-08 10:08:36,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:08:36,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:08:36,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:08:36,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d386142214054febe395e2 HTTP/1.1" 200 None 2026-04-08 10:08:36,371 - lava_callback - ERROR - Error processing callback for node 69d386142214054febe395e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50570 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:09:36,418 - lava_callback - INFO - Received callback for node 69d386562214054febe39617 INFO: 172.20.0.1:49956 - "POST /node/69d386562214054febe39617 HTTP/1.0" 202 Accepted 2026-04-08 10:09:36,424 - lava_callback - INFO - Starting async job submit for node 69d386562214054febe39617 2026-04-08 10:09:36,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:36,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d386562214054febe39617 HTTP/1.1" 200 None 2026-04-08 10:09:36,469 - lava_callback - DEBUG - Processing node 69d386562214054febe39617: baseline-arm64-baylibre 2026-04-08 10:09:36,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d386562214054febe39617 2026-04-08 10:09:36,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:36,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d386562214054febe39617 HTTP/1.1" 200 None 2026-04-08 10:09:36,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:36,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:09:36,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:36,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d386562214054febe39617 HTTP/1.1" 200 None 2026-04-08 10:09:36,535 - lava_callback - ERROR - Error processing callback for node 69d386562214054febe39617: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:09:48,201 - lava_callback - INFO - Received callback for node 69d625304b6493d86166352d INFO: 172.20.0.1:35558 - "POST /node/69d625304b6493d86166352d HTTP/1.0" 202 Accepted 2026-04-08 10:09:48,208 - lava_callback - INFO - Starting async job submit for node 69d625304b6493d86166352d 2026-04-08 10:09:48,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:48,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d625304b6493d86166352d HTTP/1.1" 200 None 2026-04-08 10:09:48,343 - lava_callback - DEBUG - Processing node 69d625304b6493d86166352d: baseline-arm 2026-04-08 10:09:48,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:09:49,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:09:49,092 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d625304b6493d86166352d/log.txt.gz 2026-04-08 10:09:49,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:09:49,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:09:49,720 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d625304b6493d86166352d/lava-logs.yaml 2026-04-08 10:09:49,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:09:49,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:09:49,906 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d625304b6493d86166352d/lava_callback.json.gz DEBUG: Setup failed for 69d625304b6493d86166352d. Transitting job node result: incomplete -> fail 2026-04-08 10:09:50,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:50,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d625304b6493d86166352d HTTP/1.1" 200 None 2026-04-08 10:09:50,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:50,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 10:09:50,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:50,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d625304b6493d86166352d HTTP/1.1" 200 None 2026-04-08 10:09:50,202 - lava_callback - INFO - Completed processing callback for node 69d625304b6493d86166352d INFO: 172.20.0.1:35892 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:09:57,042 - lava_callback - INFO - Received callback for node 69d629324b6493d8616637e1 INFO: 172.20.0.1:35904 - "POST /node/69d629324b6493d8616637e1 HTTP/1.0" 202 Accepted 2026-04-08 10:09:57,048 - lava_callback - INFO - Starting async job submit for node 69d629324b6493d8616637e1 2026-04-08 10:09:57,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:57,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d629324b6493d8616637e1 HTTP/1.1" 200 None 2026-04-08 10:09:57,092 - lava_callback - DEBUG - Processing node 69d629324b6493d8616637e1: baseline-arm64-broonie 2026-04-08 10:09:57,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:09:57,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:09:57,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d629324b6493d8616637e1/log.txt.gz 2026-04-08 10:09:57,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:09:57,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:09:57,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d629324b6493d8616637e1/lava-logs.yaml 2026-04-08 10:09:57,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:09:58,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:09:58,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d629324b6493d8616637e1/lava_callback.json.gz 2026-04-08 10:09:58,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:58,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d629324b6493d8616637e1 HTTP/1.1" 200 None 2026-04-08 10:09:58,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:58,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:09:58,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:09:58,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d629324b6493d8616637e1 HTTP/1.1" 200 None 2026-04-08 10:09:58,190 - lava_callback - INFO - Completed processing callback for node 69d629324b6493d8616637e1 2026-04-08 10:10:05,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:05,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41222 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:10:12,837 - lava_callback - INFO - Received callback for node 69d6296b4b6493d86166381d INFO: 172.20.0.1:58406 - "POST /node/69d6296b4b6493d86166381d HTTP/1.0" 202 Accepted 2026-04-08 10:10:12,843 - lava_callback - INFO - Starting async job submit for node 69d6296b4b6493d86166381d 2026-04-08 10:10:12,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:12,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6296b4b6493d86166381d HTTP/1.1" 200 None 2026-04-08 10:10:12,869 - lava_callback - DEBUG - Processing node 69d6296b4b6493d86166381d: baseline-nfs-arm64 2026-04-08 10:10:12,871 - lava_callback - WARNING - Failed to upload log 2026-04-08 10:10:12,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:10:12,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:10:12,987 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6296b4b6493d86166381d/lava-logs.yaml 2026-04-08 10:10:12,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:10:13,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:10:13,110 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6296b4b6493d86166381d/lava_callback.json.gz 2026-04-08 10:10:13,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:13,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6296b4b6493d86166381d HTTP/1.1" 200 None 2026-04-08 10:10:13,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:13,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:10:13,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:13,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6296b4b6493d86166381d HTTP/1.1" 200 None 2026-04-08 10:10:13,187 - lava_callback - INFO - Completed processing callback for node 69d6296b4b6493d86166381d INFO: 172.20.0.1:58414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49172 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:10:35,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:35,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:10:54,221 - lava_callback - INFO - Received callback for node 69d386a42214054febe39650 INFO: 172.20.0.1:36806 - "POST /node/69d386a42214054febe39650 HTTP/1.0" 202 Accepted 2026-04-08 10:10:54,227 - lava_callback - INFO - Starting async job submit for node 69d386a42214054febe39650 2026-04-08 10:10:54,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:54,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d386a42214054febe39650 HTTP/1.1" 200 None 2026-04-08 10:10:54,244 - lava_callback - DEBUG - Processing node 69d386a42214054febe39650: baseline-arm64-clabbe 2026-04-08 10:10:54,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d386a42214054febe39650 2026-04-08 10:10:54,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:54,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d386a42214054febe39650 HTTP/1.1" 200 None 2026-04-08 10:10:54,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:54,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:10:54,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:10:54,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d386a42214054febe39650 HTTP/1.1" 200 None 2026-04-08 10:10:54,315 - lava_callback - ERROR - Error processing callback for node 69d386a42214054febe39650: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49048 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:13:14,287 - lava_callback - INFO - Received callback for node 69d387312214054febe396a3 INFO: 172.20.0.1:49142 - "POST /node/69d387312214054febe396a3 HTTP/1.0" 202 Accepted 2026-04-08 10:13:14,293 - lava_callback - INFO - Starting async job submit for node 69d387312214054febe396a3 2026-04-08 10:13:14,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:14,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d387312214054febe396a3 HTTP/1.1" 200 None 2026-04-08 10:13:14,309 - lava_callback - DEBUG - Processing node 69d387312214054febe396a3: baseline-arm64-clabbe 2026-04-08 10:13:14,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d387312214054febe396a3 2026-04-08 10:13:14,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:14,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d387312214054febe396a3 HTTP/1.1" 200 None 2026-04-08 10:13:14,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:14,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:13:14,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:14,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d387312214054febe396a3 HTTP/1.1" 200 None 2026-04-08 10:13:14,366 - lava_callback - ERROR - Error processing callback for node 69d387312214054febe396a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49156 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:13:25,418 - lava_callback - INFO - Received callback for node 69d62a054b6493d8616638d3 INFO: 172.20.0.1:47596 - "POST /node/69d62a054b6493d8616638d3 HTTP/1.0" 202 Accepted 2026-04-08 10:13:25,425 - lava_callback - INFO - Starting async job submit for node 69d62a054b6493d8616638d3 2026-04-08 10:13:25,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:25,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62a054b6493d8616638d3 HTTP/1.1" 200 None 2026-04-08 10:13:25,471 - lava_callback - DEBUG - Processing node 69d62a054b6493d8616638d3: baseline-arm64 2026-04-08 10:13:25,668 - lava_callback - WARNING - Failed to upload log 2026-04-08 10:13:25,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:13:26,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:13:26,047 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62a054b6493d8616638d3/lava-logs.yaml 2026-04-08 10:13:26,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:13:26,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:13:26,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62a054b6493d8616638d3/lava_callback.json.gz 2026-04-08 10:13:26,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:26,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62a054b6493d8616638d3 HTTP/1.1" 200 None 2026-04-08 10:13:26,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:26,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:13:26,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:26,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62a054b6493d8616638d3 HTTP/1.1" 200 None 2026-04-08 10:13:26,334 - lava_callback - INFO - Completed processing callback for node 69d62a054b6493d8616638d3 INFO: 172.20.0.1:47610 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:13:35,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:13:35,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54568 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:14:03,135 - lava_callback - INFO - Received callback for node 69d62a454b6493d86166391a INFO: 172.20.0.1:40836 - "POST /node/69d62a454b6493d86166391a HTTP/1.0" 202 Accepted 2026-04-08 10:14:03,142 - lava_callback - INFO - Starting async job submit for node 69d62a454b6493d86166391a 2026-04-08 10:14:03,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:03,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62a454b6493d86166391a HTTP/1.1" 200 None 2026-04-08 10:14:03,187 - lava_callback - DEBUG - Processing node 69d62a454b6493d86166391a: baseline-arm64 2026-04-08 10:14:03,353 - lava_callback - WARNING - Failed to upload log 2026-04-08 10:14:03,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:14:03,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:14:03,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62a454b6493d86166391a/lava-logs.yaml 2026-04-08 10:14:03,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:14:03,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:14:03,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62a454b6493d86166391a/lava_callback.json.gz 2026-04-08 10:14:03,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:03,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62a454b6493d86166391a HTTP/1.1" 200 None 2026-04-08 10:14:03,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:03,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:14:03,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:03,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62a454b6493d86166391a HTTP/1.1" 200 None 2026-04-08 10:14:03,844 - lava_callback - INFO - Completed processing callback for node 69d62a454b6493d86166391a 2026-04-08 10:14:05,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:05,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54900 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:14:34,335 - lava_callback - INFO - Received callback for node 69d387792214054febe396bf INFO: 172.20.0.1:50302 - "POST /node/69d387792214054febe396bf HTTP/1.0" 202 Accepted 2026-04-08 10:14:34,341 - lava_callback - INFO - Starting async job submit for node 69d387792214054febe396bf 2026-04-08 10:14:34,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:34,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d387792214054febe396bf HTTP/1.1" 200 None 2026-04-08 10:14:34,357 - lava_callback - DEBUG - Processing node 69d387792214054febe396bf: baseline-arm64-clabbe 2026-04-08 10:14:34,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d387792214054febe396bf 2026-04-08 10:14:34,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:34,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d387792214054febe396bf HTTP/1.1" 200 None 2026-04-08 10:14:34,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:34,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:14:34,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:34,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d387792214054febe396bf HTTP/1.1" 200 None 2026-04-08 10:14:34,424 - lava_callback - ERROR - Error processing callback for node 69d387792214054febe396bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50314 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:14:37,214 - lava_callback - INFO - Received callback for node 69d629b84b6493d861663865 INFO: 172.20.0.1:50330 - "POST /node/69d629b84b6493d861663865 HTTP/1.0" 202 Accepted 2026-04-08 10:14:37,220 - lava_callback - INFO - Starting async job submit for node 69d629b84b6493d861663865 2026-04-08 10:14:37,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:37,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d629b84b6493d861663865 HTTP/1.1" 200 None 2026-04-08 10:14:37,257 - lava_callback - DEBUG - Processing node 69d629b84b6493d861663865: baseline-arm64-broonie 2026-04-08 10:14:37,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:14:37,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:14:37,656 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d629b84b6493d861663865/log.txt.gz 2026-04-08 10:14:37,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:14:38,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:14:38,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d629b84b6493d861663865/lava-logs.yaml 2026-04-08 10:14:38,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:14:38,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:14:38,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d629b84b6493d861663865/lava_callback.json.gz 2026-04-08 10:14:38,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:38,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d629b84b6493d861663865 HTTP/1.1" 200 None 2026-04-08 10:14:38,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:38,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:14:38,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:14:38,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d629b84b6493d861663865 HTTP/1.1" 200 None 2026-04-08 10:14:38,302 - lava_callback - INFO - Completed processing callback for node 69d629b84b6493d861663865 INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46048 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:15:05,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:15:05,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45964 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:17:31,617 - lava_callback - INFO - Received callback for node 69d62acf4b6493d8616639c0 INFO: 172.20.0.1:33800 - "POST /node/69d62acf4b6493d8616639c0 HTTP/1.0" 202 Accepted 2026-04-08 10:17:31,623 - lava_callback - INFO - Starting async job submit for node 69d62acf4b6493d8616639c0 2026-04-08 10:17:31,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:31,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62acf4b6493d8616639c0 HTTP/1.1" 200 None 2026-04-08 10:17:31,668 - lava_callback - DEBUG - Processing node 69d62acf4b6493d8616639c0: baseline-arm64 2026-04-08 10:17:31,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:17:32,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:17:32,130 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62acf4b6493d8616639c0/log.txt.gz 2026-04-08 10:17:32,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:17:32,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:17:32,546 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62acf4b6493d8616639c0/lava-logs.yaml 2026-04-08 10:17:32,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:17:32,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:17:32,721 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62acf4b6493d8616639c0/lava_callback.json.gz 2026-04-08 10:17:32,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:32,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62acf4b6493d8616639c0 HTTP/1.1" 200 None 2026-04-08 10:17:32,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:32,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:17:32,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:32,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62acf4b6493d8616639c0 HTTP/1.1" 200 None 2026-04-08 10:17:32,821 - lava_callback - INFO - Completed processing callback for node 69d62acf4b6493d8616639c0 2026-04-08 10:17:35,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:35,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44212 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:17:54,546 - lava_callback - INFO - Received callback for node 69d388492214054febe39767 INFO: 172.20.0.1:45168 - "POST /node/69d388492214054febe39767 HTTP/1.0" 202 Accepted 2026-04-08 10:17:54,551 - lava_callback - INFO - Starting async job submit for node 69d388492214054febe39767 2026-04-08 10:17:54,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:54,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d388492214054febe39767 HTTP/1.1" 200 None 2026-04-08 10:17:54,568 - lava_callback - DEBUG - Processing node 69d388492214054febe39767: baseline-arm64-clabbe 2026-04-08 10:17:54,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d388492214054febe39767 2026-04-08 10:17:54,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:54,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d388492214054febe39767 HTTP/1.1" 200 None 2026-04-08 10:17:54,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:54,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:17:54,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:17:54,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d388492214054febe39767 HTTP/1.1" 200 None 2026-04-08 10:17:54,630 - lava_callback - ERROR - Error processing callback for node 69d388492214054febe39767: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:21:17,429 - lava_callback - INFO - Received callback for node 69d3890b2214054febe3980e INFO: 172.20.0.1:46472 - "POST /node/69d3890b2214054febe3980e HTTP/1.0" 202 Accepted 2026-04-08 10:21:17,435 - lava_callback - INFO - Starting async job submit for node 69d3890b2214054febe3980e 2026-04-08 10:21:17,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:17,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3890b2214054febe3980e HTTP/1.1" 200 None 2026-04-08 10:21:17,451 - lava_callback - DEBUG - Processing node 69d3890b2214054febe3980e: baseline-arm64-baylibre 2026-04-08 10:21:17,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d3890b2214054febe3980e 2026-04-08 10:21:17,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:17,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d3890b2214054febe3980e HTTP/1.1" 200 None 2026-04-08 10:21:17,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:17,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:21:17,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:17,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d3890b2214054febe3980e HTTP/1.1" 200 None 2026-04-08 10:21:17,516 - lava_callback - ERROR - Error processing callback for node 69d3890b2214054febe3980e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46520 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:21:39,239 - lava_callback - INFO - Received callback for node 69d6279d4b6493d86166371a INFO: 172.20.0.1:46536 - "POST /node/69d6279d4b6493d86166371a HTTP/1.0" 202 Accepted 2026-04-08 10:21:39,247 - lava_callback - INFO - Starting async job submit for node 69d6279d4b6493d86166371a 2026-04-08 10:21:39,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:39,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6279d4b6493d86166371a HTTP/1.1" 200 None 2026-04-08 10:21:39,314 - lava_callback - DEBUG - Processing node 69d6279d4b6493d86166371a: baseline-nfs-arm64 2026-04-08 10:21:40,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51572 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:21:40,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:21:40,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6279d4b6493d86166371a/log.txt.gz 2026-04-08 10:21:41,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:21:41,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:21:41,396 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6279d4b6493d86166371a/lava-logs.yaml 2026-04-08 10:21:41,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:21:41,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:21:41,773 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6279d4b6493d86166371a/lava_callback.json.gz DEBUG: Setup failed for 69d6279d4b6493d86166371a. Transitting job node result: incomplete -> fail 2026-04-08 10:21:41,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:41,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6279d4b6493d86166371a HTTP/1.1" 200 None 2026-04-08 10:21:41,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:41,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:21:41,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:21:41,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6279d4b6493d86166371a HTTP/1.1" 200 None 2026-04-08 10:21:42,044 - lava_callback - INFO - Completed processing callback for node 69d6279d4b6493d86166371a INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:22:05,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:22:05,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57872 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:22:50,517 - lava_callback - INFO - Received callback for node 69d62c344b6493d861663b3a INFO: 172.20.0.1:59746 - "POST /node/69d62c344b6493d861663b3a HTTP/1.0" 202 Accepted 2026-04-08 10:22:50,524 - lava_callback - INFO - Starting async job submit for node 69d62c344b6493d861663b3a 2026-04-08 10:22:50,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:22:50,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62c344b6493d861663b3a HTTP/1.1" 200 None 2026-04-08 10:22:50,563 - lava_callback - DEBUG - Processing node 69d62c344b6493d861663b3a: baseline-arm64 2026-04-08 10:22:50,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:22:51,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:22:51,052 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62c344b6493d861663b3a/log.txt.gz 2026-04-08 10:22:51,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:22:51,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:22:51,615 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62c344b6493d861663b3a/lava-logs.yaml 2026-04-08 10:22:51,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:22:51,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:22:51,963 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62c344b6493d861663b3a/lava_callback.json.gz 2026-04-08 10:22:52,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:22:52,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62c344b6493d861663b3a HTTP/1.1" 200 None 2026-04-08 10:22:52,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:22:52,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:22:52,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:22:52,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62c344b6493d861663b3a HTTP/1.1" 200 None 2026-04-08 10:22:52,122 - lava_callback - INFO - Completed processing callback for node 69d62c344b6493d861663b3a INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:23:05,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:05,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37156 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:23:36,611 - lava_callback - INFO - Received callback for node 69d62bc44b6493d861663ac8 INFO: 172.20.0.1:37162 - "POST /node/69d62bc44b6493d861663ac8 HTTP/1.0" 202 Accepted 2026-04-08 10:23:36,618 - lava_callback - INFO - Starting async job submit for node 69d62bc44b6493d861663ac8 2026-04-08 10:23:36,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:36,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62bc44b6493d861663ac8 HTTP/1.1" 200 None 2026-04-08 10:23:36,658 - lava_callback - DEBUG - Processing node 69d62bc44b6493d861663ac8: ltp-fs_bind 2026-04-08 10:23:37,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:23:37,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:23:37,156 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69d62bc44b6493d861663ac8/log.txt.gz 2026-04-08 10:23:37,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:23:37,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:23:37,570 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69d62bc44b6493d861663ac8/lava-logs.yaml 2026-04-08 10:23:37,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:23:37,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:23:37,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69d62bc44b6493d861663ac8/lava_callback.json.gz 2026-04-08 10:23:37,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:37,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62bc44b6493d861663ac8 HTTP/1.1" 200 None 2026-04-08 10:23:37,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:37,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 10:23:37,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:37,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62bc44b6493d861663ac8 HTTP/1.1" 200 None 2026-04-08 10:23:37,885 - lava_callback - INFO - Completed processing callback for node 69d62bc44b6493d861663ac8 INFO: 172.20.0.1:46044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56106 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:23:56,597 - lava_callback - INFO - Received callback for node 69d62be94b6493d861663af1 INFO: 172.20.0.1:56118 - "POST /node/69d62be94b6493d861663af1 HTTP/1.0" 202 Accepted 2026-04-08 10:23:56,604 - lava_callback - INFO - Starting async job submit for node 69d62be94b6493d861663af1 2026-04-08 10:23:56,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:56,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62be94b6493d861663af1 HTTP/1.1" 200 None 2026-04-08 10:23:56,651 - lava_callback - DEBUG - Processing node 69d62be94b6493d861663af1: ltp-syscalls 2026-04-08 10:23:57,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:23:57,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:23:57,520 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69d62be94b6493d861663af1/log.txt.gz 2026-04-08 10:23:57,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:23:58,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:23:58,131 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69d62be94b6493d861663af1/lava-logs.yaml 2026-04-08 10:23:58,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:23:58,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:23:58,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69d62be94b6493d861663af1/lava_callback.json.gz 2026-04-08 10:23:58,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:58,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62be94b6493d861663af1 HTTP/1.1" 200 None 2026-04-08 10:23:58,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:58,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 10:23:58,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:23:58,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62be94b6493d861663af1 HTTP/1.1" 200 None 2026-04-08 10:23:58,399 - lava_callback - INFO - Completed processing callback for node 69d62be94b6493d861663af1 2026-04-08 10:24:05,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:24:05,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53474 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:24:37,177 - lava_callback - INFO - Received callback for node 69d389d82214054febe398a7 INFO: 172.20.0.1:53484 - "POST /node/69d389d82214054febe398a7 HTTP/1.0" 202 Accepted 2026-04-08 10:24:37,185 - lava_callback - INFO - Starting async job submit for node 69d389d82214054febe398a7 2026-04-08 10:24:37,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:24:37,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d389d82214054febe398a7 HTTP/1.1" 200 None 2026-04-08 10:24:37,203 - lava_callback - DEBUG - Processing node 69d389d82214054febe398a7: baseline-arm64-baylibre 2026-04-08 10:24:37,203 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d389d82214054febe398a7 2026-04-08 10:24:37,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:24:37,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d389d82214054febe398a7 HTTP/1.1" 200 None 2026-04-08 10:24:37,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:24:37,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:24:37,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:24:37,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d389d82214054febe398a7 HTTP/1.1" 200 None 2026-04-08 10:24:37,266 - lava_callback - ERROR - Error processing callback for node 69d389d82214054febe398a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38966 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:25:57,268 - lava_callback - INFO - Received callback for node 69d38a222214054febe398cb INFO: 172.20.0.1:38974 - "POST /node/69d38a222214054febe398cb HTTP/1.0" 202 Accepted 2026-04-08 10:25:57,274 - lava_callback - INFO - Starting async job submit for node 69d38a222214054febe398cb 2026-04-08 10:25:57,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:25:57,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38a222214054febe398cb HTTP/1.1" 200 None 2026-04-08 10:25:57,289 - lava_callback - DEBUG - Processing node 69d38a222214054febe398cb: baseline-arm64-baylibre 2026-04-08 10:25:57,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38a222214054febe398cb 2026-04-08 10:25:57,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:25:57,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38a222214054febe398cb HTTP/1.1" 200 None 2026-04-08 10:25:57,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:25:57,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:25:57,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:25:57,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38a222214054febe398cb HTTP/1.1" 200 None 2026-04-08 10:25:57,398 - lava_callback - ERROR - Error processing callback for node 69d38a222214054febe398cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33248 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:26:14,863 - lava_callback - INFO - Received callback for node 69d62cf04b6493d861663bdc INFO: 172.20.0.1:46880 - "POST /node/69d62cf04b6493d861663bdc HTTP/1.0" 202 Accepted 2026-04-08 10:26:14,869 - lava_callback - INFO - Starting async job submit for node 69d62cf04b6493d861663bdc 2026-04-08 10:26:14,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:14,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62cf04b6493d861663bdc HTTP/1.1" 200 None 2026-04-08 10:26:14,909 - lava_callback - DEBUG - Processing node 69d62cf04b6493d861663bdc: baseline-x86 2026-04-08 10:26:15,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:46894 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:26:15,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:26:15,301 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d62cf04b6493d861663bdc/log.txt.gz 2026-04-08 10:26:15,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:26:15,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:26:15,692 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d62cf04b6493d861663bdc/lava-logs.yaml 2026-04-08 10:26:15,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:26:16,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:26:16,024 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d62cf04b6493d861663bdc/lava_callback.json.gz 2026-04-08 10:26:16,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:16,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62cf04b6493d861663bdc HTTP/1.1" 200 None 2026-04-08 10:26:16,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:16,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f334b6493d861661fc9 HTTP/1.1" 200 None 2026-04-08 10:26:16,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:16,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62cf04b6493d861663bdc HTTP/1.1" 200 None 2026-04-08 10:26:16,108 - lava_callback - INFO - Completed processing callback for node 69d62cf04b6493d861663bdc INFO: 172.20.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43392 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:26:35,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:35,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:26:57,384 - lava_callback - INFO - Received callback for node 69d38a6e2214054febe398d3 INFO: 172.20.0.1:50176 - "POST /node/69d38a6e2214054febe398d3 HTTP/1.0" 202 Accepted 2026-04-08 10:26:57,390 - lava_callback - INFO - Starting async job submit for node 69d38a6e2214054febe398d3 2026-04-08 10:26:57,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:57,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38a6e2214054febe398d3 HTTP/1.1" 200 None 2026-04-08 10:26:57,407 - lava_callback - DEBUG - Processing node 69d38a6e2214054febe398d3: baseline-arm64-baylibre 2026-04-08 10:26:57,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38a6e2214054febe398d3 2026-04-08 10:26:57,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:57,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38a6e2214054febe398d3 HTTP/1.1" 200 None 2026-04-08 10:26:57,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:57,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:26:57,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:26:57,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38a6e2214054febe398d3 HTTP/1.1" 200 None 2026-04-08 10:26:57,467 - lava_callback - ERROR - Error processing callback for node 69d38a6e2214054febe398d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:27:53,438 - lava_callback - INFO - Received callback for node 69d62d9d4b6493d861663c9c INFO: 172.20.0.1:35822 - "POST /node/69d62d9d4b6493d861663c9c HTTP/1.0" 202 Accepted 2026-04-08 10:27:53,444 - lava_callback - INFO - Starting async job submit for node 69d62d9d4b6493d861663c9c 2026-04-08 10:27:53,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:27:53,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62d9d4b6493d861663c9c HTTP/1.1" 200 None 2026-04-08 10:27:53,476 - lava_callback - DEBUG - Processing node 69d62d9d4b6493d861663c9c: baseline-nfs-arm64 2026-04-08 10:27:53,477 - lava_callback - WARNING - Failed to upload log 2026-04-08 10:27:53,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:27:53,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:27:53,626 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d62d9d4b6493d861663c9c/lava-logs.yaml 2026-04-08 10:27:53,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:27:53,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:27:53,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d62d9d4b6493d861663c9c/lava_callback.json.gz 2026-04-08 10:27:53,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:27:53,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62d9d4b6493d861663c9c HTTP/1.1" 200 None 2026-04-08 10:27:53,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:27:53,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:27:53,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:27:53,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62d9d4b6493d861663c9c HTTP/1.1" 200 None 2026-04-08 10:27:53,822 - lava_callback - INFO - Completed processing callback for node 69d62d9d4b6493d861663c9c INFO: 172.20.0.1:35838 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:28:01,091 - lava_callback - INFO - Received callback for node 69d62bf24b6493d861663af4 INFO: 172.20.0.1:46564 - "POST /node/69d62bf24b6493d861663af4 HTTP/1.0" 202 Accepted 2026-04-08 10:28:01,099 - lava_callback - INFO - Starting async job submit for node 69d62bf24b6493d861663af4 2026-04-08 10:28:01,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:01,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62bf24b6493d861663af4 HTTP/1.1" 200 None 2026-04-08 10:28:01,151 - lava_callback - DEBUG - Processing node 69d62bf24b6493d861663af4: baseline-arm64 2026-04-08 10:28:01,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:28:01,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:28:01,648 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62bf24b6493d861663af4/log.txt.gz 2026-04-08 10:28:01,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:28:02,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:28:02,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62bf24b6493d861663af4/lava-logs.yaml 2026-04-08 10:28:02,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:28:02,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:28:02,298 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62bf24b6493d861663af4/lava_callback.json.gz DEBUG: Setup failed for 69d62bf24b6493d861663af4. Transitting job node result: incomplete -> fail 2026-04-08 10:28:02,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:02,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62bf24b6493d861663af4 HTTP/1.1" 200 None 2026-04-08 10:28:02,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:02,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:28:02,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:02,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62bf24b6493d861663af4 HTTP/1.1" 200 None 2026-04-08 10:28:02,504 - lava_callback - INFO - Completed processing callback for node 69d62bf24b6493d861663af4 2026-04-08 10:28:05,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60252 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:28:17,400 - lava_callback - INFO - Received callback for node 69d38ab52214054febe39914 INFO: 172.20.0.1:60266 - "POST /node/69d38ab52214054febe39914 HTTP/1.0" 202 Accepted 2026-04-08 10:28:17,406 - lava_callback - INFO - Starting async job submit for node 69d38ab52214054febe39914 2026-04-08 10:28:17,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:17,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38ab52214054febe39914 HTTP/1.1" 200 None 2026-04-08 10:28:17,430 - lava_callback - DEBUG - Processing node 69d38ab52214054febe39914: baseline-arm64-baylibre 2026-04-08 10:28:17,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38ab52214054febe39914 2026-04-08 10:28:17,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:17,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38ab52214054febe39914 HTTP/1.1" 200 None 2026-04-08 10:28:17,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:17,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:28:17,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:28:17,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38ab52214054febe39914 HTTP/1.1" 200 None 2026-04-08 10:28:17,488 - lava_callback - ERROR - Error processing callback for node 69d38ab52214054febe39914: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:29:17,406 - lava_callback - INFO - Received callback for node 69d38afb2214054febe39940 INFO: 172.20.0.1:50756 - "POST /node/69d38afb2214054febe39940 HTTP/1.0" 202 Accepted 2026-04-08 10:29:17,411 - lava_callback - INFO - Starting async job submit for node 69d38afb2214054febe39940 2026-04-08 10:29:17,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:29:17,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38afb2214054febe39940 HTTP/1.1" 200 None 2026-04-08 10:29:17,427 - lava_callback - DEBUG - Processing node 69d38afb2214054febe39940: baseline-arm64-baylibre 2026-04-08 10:29:17,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38afb2214054febe39940 2026-04-08 10:29:17,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:29:17,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38afb2214054febe39940 HTTP/1.1" 200 None 2026-04-08 10:29:17,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:29:17,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:29:17,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:29:17,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38afb2214054febe39940 HTTP/1.1" 200 None 2026-04-08 10:29:17,512 - lava_callback - ERROR - Error processing callback for node 69d38afb2214054febe39940: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57682 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:29:59,227 - lava_callback - INFO - Received callback for node 69d62ddf4b6493d861663cb0 INFO: 172.20.0.1:57690 - "POST /node/69d62ddf4b6493d861663cb0 HTTP/1.0" 202 Accepted 2026-04-08 10:29:59,244 - lava_callback - INFO - Starting async job submit for node 69d62ddf4b6493d861663cb0 2026-04-08 10:29:59,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:29:59,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62ddf4b6493d861663cb0 HTTP/1.1" 200 None 2026-04-08 10:29:59,340 - lava_callback - DEBUG - Processing node 69d62ddf4b6493d861663cb0: baseline-arm64 2026-04-08 10:29:59,552 - lava_callback - WARNING - Failed to upload log 2026-04-08 10:29:59,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:29:59,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:29:59,899 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62ddf4b6493d861663cb0/lava-logs.yaml 2026-04-08 10:29:59,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:30:00,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:30:00,049 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62ddf4b6493d861663cb0/lava_callback.json.gz 2026-04-08 10:30:00,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:00,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62ddf4b6493d861663cb0 HTTP/1.1" 200 None 2026-04-08 10:30:00,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:00,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:30:00,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:00,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62ddf4b6493d861663cb0 HTTP/1.1" 200 None 2026-04-08 10:30:00,218 - lava_callback - INFO - Completed processing callback for node 69d62ddf4b6493d861663cb0 2026-04-08 10:30:05,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:06,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37128 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:30:54,174 - lava_callback - INFO - Received callback for node 69d62e264b6493d861663cc5 INFO: 172.20.0.1:45808 - "POST /node/69d62e264b6493d861663cc5 HTTP/1.0" 202 Accepted 2026-04-08 10:30:54,181 - lava_callback - INFO - Starting async job submit for node 69d62e264b6493d861663cc5 2026-04-08 10:30:54,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:54,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62e264b6493d861663cc5 HTTP/1.1" 200 None 2026-04-08 10:30:54,234 - lava_callback - DEBUG - Processing node 69d62e264b6493d861663cc5: baseline-arm64 2026-04-08 10:30:54,367 - lava_callback - WARNING - Failed to upload log 2026-04-08 10:30:54,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:30:54,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:30:54,603 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62e264b6493d861663cc5/lava-logs.yaml 2026-04-08 10:30:54,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:30:54,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:30:54,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62e264b6493d861663cc5/lava_callback.json.gz 2026-04-08 10:30:54,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:54,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62e264b6493d861663cc5 HTTP/1.1" 200 None 2026-04-08 10:30:54,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:54,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:30:54,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:54,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62e264b6493d861663cc5 HTTP/1.1" 200 None 2026-04-08 10:30:54,922 - lava_callback - INFO - Completed processing callback for node 69d62e264b6493d861663cc5 2026-04-08 10:30:55,076 - lava_callback - INFO - Received callback for node 69d38b4b2214054febe3995b INFO: 172.20.0.1:45812 - "POST /node/69d38b4b2214054febe3995b HTTP/1.0" 202 Accepted 2026-04-08 10:30:55,082 - lava_callback - INFO - Starting async job submit for node 69d38b4b2214054febe3995b 2026-04-08 10:30:55,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:55,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38b4b2214054febe3995b HTTP/1.1" 200 None 2026-04-08 10:30:55,100 - lava_callback - DEBUG - Processing node 69d38b4b2214054febe3995b: baseline-arm64-clabbe 2026-04-08 10:30:55,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38b4b2214054febe3995b 2026-04-08 10:30:55,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:55,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38b4b2214054febe3995b HTTP/1.1" 200 None 2026-04-08 10:30:55,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:55,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:30:55,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:30:55,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38b4b2214054febe3995b HTTP/1.1" 200 None 2026-04-08 10:30:55,161 - lava_callback - ERROR - Error processing callback for node 69d38b4b2214054febe3995b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45822 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:31:06,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:31:06,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55804 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:31:55,122 - lava_callback - INFO - Received callback for node 69d38b8a2214054febe399b8 INFO: 172.20.0.1:40374 - "POST /node/69d38b8a2214054febe399b8 HTTP/1.0" 202 Accepted 2026-04-08 10:31:55,128 - lava_callback - INFO - Starting async job submit for node 69d38b8a2214054febe399b8 2026-04-08 10:31:55,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:31:55,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38b8a2214054febe399b8 HTTP/1.1" 200 None 2026-04-08 10:31:55,144 - lava_callback - DEBUG - Processing node 69d38b8a2214054febe399b8: baseline-arm64-clabbe 2026-04-08 10:31:55,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38b8a2214054febe399b8 2026-04-08 10:31:55,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:31:55,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38b8a2214054febe399b8 HTTP/1.1" 200 None 2026-04-08 10:31:55,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:31:55,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:31:55,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:31:55,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38b8a2214054febe399b8 HTTP/1.1" 200 None 2026-04-08 10:31:55,203 - lava_callback - ERROR - Error processing callback for node 69d38b8a2214054febe399b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34608 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:34:15,585 - lava_callback - INFO - Received callback for node 69d38c1a2214054febe399f5 INFO: 172.20.0.1:34618 - "POST /node/69d38c1a2214054febe399f5 HTTP/1.0" 202 Accepted 2026-04-08 10:34:15,591 - lava_callback - INFO - Starting async job submit for node 69d38c1a2214054febe399f5 2026-04-08 10:34:15,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:34:15,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38c1a2214054febe399f5 HTTP/1.1" 200 None 2026-04-08 10:34:15,612 - lava_callback - DEBUG - Processing node 69d38c1a2214054febe399f5: baseline-arm64-clabbe 2026-04-08 10:34:15,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38c1a2214054febe399f5 2026-04-08 10:34:15,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:34:15,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38c1a2214054febe399f5 HTTP/1.1" 200 None 2026-04-08 10:34:15,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:34:15,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:34:15,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:34:15,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38c1a2214054febe399f5 HTTP/1.1" 200 None 2026-04-08 10:34:15,678 - lava_callback - ERROR - Error processing callback for node 69d38c1a2214054febe399f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53076 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:35:16,439 - lava_callback - INFO - Received callback for node 69d62eec4b6493d861663db7 INFO: 172.20.0.1:53080 - "POST /node/69d62eec4b6493d861663db7 HTTP/1.0" 202 Accepted 2026-04-08 10:35:16,445 - lava_callback - INFO - Starting async job submit for node 69d62eec4b6493d861663db7 2026-04-08 10:35:16,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:35:16,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62eec4b6493d861663db7 HTTP/1.1" 200 None 2026-04-08 10:35:16,482 - lava_callback - DEBUG - Processing node 69d62eec4b6493d861663db7: baseline-arm64 INFO: 172.20.0.1:53094 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:35:16,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:35:16,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:35:16,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62eec4b6493d861663db7/log.txt.gz 2026-04-08 10:35:17,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:35:17,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:35:17,508 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62eec4b6493d861663db7/lava-logs.yaml 2026-04-08 10:35:17,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:35:17,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:35:17,660 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d62eec4b6493d861663db7/lava_callback.json.gz 2026-04-08 10:35:17,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:35:17,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62eec4b6493d861663db7 HTTP/1.1" 200 None 2026-04-08 10:35:17,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:35:17,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:35:17,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:35:17,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62eec4b6493d861663db7 HTTP/1.1" 200 None 2026-04-08 10:35:17,825 - lava_callback - INFO - Completed processing callback for node 69d62eec4b6493d861663db7 INFO: 172.20.0.1:46512 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:35:36,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:35:36,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:36:55,314 - lava_callback - INFO - Received callback for node 69d38cc52214054febe39a7b INFO: 172.20.0.1:46808 - "POST /node/69d38cc52214054febe39a7b HTTP/1.0" 202 Accepted 2026-04-08 10:36:55,320 - lava_callback - INFO - Starting async job submit for node 69d38cc52214054febe39a7b 2026-04-08 10:36:55,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:36:55,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38cc52214054febe39a7b HTTP/1.1" 200 None 2026-04-08 10:36:55,336 - lava_callback - DEBUG - Processing node 69d38cc52214054febe39a7b: baseline-arm64-clabbe 2026-04-08 10:36:55,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38cc52214054febe39a7b 2026-04-08 10:36:55,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:36:55,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38cc52214054febe39a7b HTTP/1.1" 200 None 2026-04-08 10:36:55,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:36:55,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:36:55,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:36:55,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38cc52214054febe39a7b HTTP/1.1" 200 None 2026-04-08 10:36:55,393 - lava_callback - ERROR - Error processing callback for node 69d38cc52214054febe39a7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33456 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:37:18,404 - lava_callback - INFO - Received callback for node 69d62bcc4b6493d861663acb INFO: 172.20.0.1:33470 - "POST /node/69d62bcc4b6493d861663acb HTTP/1.0" 202 Accepted 2026-04-08 10:37:18,411 - lava_callback - INFO - Starting async job submit for node 69d62bcc4b6493d861663acb 2026-04-08 10:37:18,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:37:18,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62bcc4b6493d861663acb HTTP/1.1" 200 None 2026-04-08 10:37:18,486 - lava_callback - DEBUG - Processing node 69d62bcc4b6493d861663acb: ltp-smoke 2026-04-08 10:37:18,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:37:19,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:37:19,281 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69d62bcc4b6493d861663acb/log.txt.gz 2026-04-08 10:37:19,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:37:19,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:37:19,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69d62bcc4b6493d861663acb/lava-logs.yaml 2026-04-08 10:37:19,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:37:19,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:37:19,970 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69d62bcc4b6493d861663acb/lava_callback.json.gz 2026-04-08 10:37:20,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:37:20,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62bcc4b6493d861663acb HTTP/1.1" 200 None 2026-04-08 10:37:20,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:37:20,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f134b6493d861661fa6 HTTP/1.1" 200 None 2026-04-08 10:37:20,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:37:20,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62bcc4b6493d861663acb HTTP/1.1" 200 None 2026-04-08 10:37:20,283 - lava_callback - INFO - Completed processing callback for node 69d62bcc4b6493d861663acb INFO: 172.20.0.1:42136 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:37:36,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:37:36,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56298 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:38:15,374 - lava_callback - INFO - Received callback for node 69d38d0b2214054febe39aa4 INFO: 172.20.0.1:56300 - "POST /node/69d38d0b2214054febe39aa4 HTTP/1.0" 202 Accepted 2026-04-08 10:38:15,379 - lava_callback - INFO - Starting async job submit for node 69d38d0b2214054febe39aa4 2026-04-08 10:38:15,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:38:15,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38d0b2214054febe39aa4 HTTP/1.1" 200 None 2026-04-08 10:38:15,395 - lava_callback - DEBUG - Processing node 69d38d0b2214054febe39aa4: baseline-arm64-clabbe 2026-04-08 10:38:15,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38d0b2214054febe39aa4 2026-04-08 10:38:15,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:38:15,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38d0b2214054febe39aa4 HTTP/1.1" 200 None 2026-04-08 10:38:15,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:38:15,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:38:15,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:38:15,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38d0b2214054febe39aa4 HTTP/1.1" 200 None 2026-04-08 10:38:15,449 - lava_callback - ERROR - Error processing callback for node 69d38d0b2214054febe39aa4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51514 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:40:35,482 - lava_callback - INFO - Received callback for node 69d38d9d2214054febe39aec INFO: 172.20.0.1:41052 - "POST /node/69d38d9d2214054febe39aec HTTP/1.0" 202 Accepted 2026-04-08 10:40:35,488 - lava_callback - INFO - Starting async job submit for node 69d38d9d2214054febe39aec 2026-04-08 10:40:35,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:40:35,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38d9d2214054febe39aec HTTP/1.1" 200 None 2026-04-08 10:40:35,505 - lava_callback - DEBUG - Processing node 69d38d9d2214054febe39aec: baseline-arm64-clabbe 2026-04-08 10:40:35,505 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38d9d2214054febe39aec 2026-04-08 10:40:35,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:40:35,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38d9d2214054febe39aec HTTP/1.1" 200 None 2026-04-08 10:40:35,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:40:35,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce2677cb08ea23a1f6807d HTTP/1.1" 200 None 2026-04-08 10:40:35,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:40:35,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38d9d2214054febe39aec HTTP/1.1" 200 None 2026-04-08 10:40:35,569 - lava_callback - ERROR - Error processing callback for node 69d38d9d2214054febe39aec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56430 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:41:55,603 - lava_callback - INFO - Received callback for node 69d38de62214054febe39b0f INFO: 172.20.0.1:34162 - "POST /node/69d38de62214054febe39b0f HTTP/1.0" 202 Accepted 2026-04-08 10:41:55,613 - lava_callback - INFO - Starting async job submit for node 69d38de62214054febe39b0f 2026-04-08 10:41:55,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:41:55,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38de62214054febe39b0f HTTP/1.1" 200 None 2026-04-08 10:41:55,636 - lava_callback - DEBUG - Processing node 69d38de62214054febe39b0f: baseline-arm64-clabbe 2026-04-08 10:41:55,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38de62214054febe39b0f 2026-04-08 10:41:55,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:41:55,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38de62214054febe39b0f HTTP/1.1" 200 None 2026-04-08 10:41:55,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:41:55,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:41:55,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:41:55,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38de62214054febe39b0f HTTP/1.1" 200 None 2026-04-08 10:41:55,714 - lava_callback - ERROR - Error processing callback for node 69d38de62214054febe39b0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37044 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:43:38,767 - lava_callback - INFO - Received callback for node 69d630114b6493d861663e47 INFO: 172.20.0.1:37054 - "POST /node/69d630114b6493d861663e47 HTTP/1.0" 202 Accepted 2026-04-08 10:43:38,774 - lava_callback - INFO - Starting async job submit for node 69d630114b6493d861663e47 2026-04-08 10:43:38,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:43:38,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d630114b6493d861663e47 HTTP/1.1" 200 None 2026-04-08 10:43:38,825 - lava_callback - DEBUG - Processing node 69d630114b6493d861663e47: baseline-arm64 2026-04-08 10:43:39,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:43:39,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:43:39,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d630114b6493d861663e47/log.txt.gz 2026-04-08 10:43:39,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:43:39,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:43:39,909 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d630114b6493d861663e47/lava-logs.yaml 2026-04-08 10:43:39,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:43:40,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:43:40,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d630114b6493d861663e47/lava_callback.json.gz DEBUG: Setup failed for 69d630114b6493d861663e47. Transitting job node result: incomplete -> fail 2026-04-08 10:43:40,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:43:40,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d630114b6493d861663e47 HTTP/1.1" 200 None 2026-04-08 10:43:40,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:43:40,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:43:40,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:43:40,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d630114b6493d861663e47 HTTP/1.1" 200 None 2026-04-08 10:43:40,278 - lava_callback - INFO - Completed processing callback for node 69d630114b6493d861663e47 INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60528 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:44:06,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:44:06,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:44:15,721 - lava_callback - INFO - Received callback for node 69d38e6d2214054febe39b8d INFO: 172.20.0.1:45160 - "POST /node/69d38e6d2214054febe39b8d HTTP/1.0" 202 Accepted 2026-04-08 10:44:15,738 - lava_callback - INFO - Starting async job submit for node 69d38e6d2214054febe39b8d 2026-04-08 10:44:15,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:44:15,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38e6d2214054febe39b8d HTTP/1.1" 200 None 2026-04-08 10:44:15,792 - lava_callback - DEBUG - Processing node 69d38e6d2214054febe39b8d: baseline-arm64-clabbe 2026-04-08 10:44:15,792 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69d38e6d2214054febe39b8d 2026-04-08 10:44:15,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:44:15,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d38e6d2214054febe39b8d HTTP/1.1" 200 None 2026-04-08 10:44:15,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:44:15,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-08 10:44:15,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:44:15,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d38e6d2214054febe39b8d HTTP/1.1" 200 None 2026-04-08 10:44:15,875 - lava_callback - ERROR - Error processing callback for node 69d38e6d2214054febe39b8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44294 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:45:03,282 - lava_callback - INFO - Received callback for node 69d6239f4b6493d861663412 INFO: 172.20.0.1:33012 - "POST /node/69d6239f4b6493d861663412 HTTP/1.0" 202 Accepted 2026-04-08 10:45:03,289 - lava_callback - INFO - Starting async job submit for node 69d6239f4b6493d861663412 2026-04-08 10:45:03,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:03,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6239f4b6493d861663412 HTTP/1.1" 200 None 2026-04-08 10:45:03,330 - lava_callback - DEBUG - Processing node 69d6239f4b6493d861663412: baseline-arm64 2026-04-08 10:45:03,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:45:03,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:45:03,982 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6239f4b6493d861663412/log.txt.gz 2026-04-08 10:45:04,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:45:04,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:45:04,476 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6239f4b6493d861663412/lava-logs.yaml 2026-04-08 10:45:04,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:45:04,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:45:04,654 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6239f4b6493d861663412/lava_callback.json.gz 2026-04-08 10:45:04,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:04,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6239f4b6493d861663412 HTTP/1.1" 200 None 2026-04-08 10:45:04,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:04,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:45:04,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:04,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6239f4b6493d861663412 HTTP/1.1" 200 None 2026-04-08 10:45:04,821 - lava_callback - INFO - Completed processing callback for node 69d6239f4b6493d861663412 2026-04-08 10:45:06,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:06,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:45:15,838 - lava_callback - INFO - Received callback for node 69d62d764b6493d861663c57 INFO: 172.20.0.1:44724 - "POST /node/69d62d764b6493d861663c57 HTTP/1.0" 202 Accepted 2026-04-08 10:45:15,845 - lava_callback - INFO - Starting async job submit for node 69d62d764b6493d861663c57 2026-04-08 10:45:15,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:15,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62d764b6493d861663c57 HTTP/1.1" 200 None 2026-04-08 10:45:15,916 - lava_callback - DEBUG - Processing node 69d62d764b6493d861663c57: baseline-arm INFO: 172.20.0.1:44734 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:45:16,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:45:16,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:45:16,718 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d62d764b6493d861663c57/log.txt.gz 2026-04-08 10:45:17,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:45:17,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:45:17,282 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d62d764b6493d861663c57/lava-logs.yaml 2026-04-08 10:45:17,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:45:17,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:45:17,441 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d62d764b6493d861663c57/lava_callback.json.gz DEBUG: Setup failed for 69d62d764b6493d861663c57. Transitting job node result: incomplete -> fail 2026-04-08 10:45:17,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:17,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62d764b6493d861663c57 HTTP/1.1" 200 None 2026-04-08 10:45:17,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:17,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 10:45:17,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:17,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62d764b6493d861663c57 HTTP/1.1" 200 None 2026-04-08 10:45:17,734 - lava_callback - INFO - Completed processing callback for node 69d62d764b6493d861663c57 INFO: 172.20.0.1:39618 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:45:36,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:45:36,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44606 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:50:41,728 - lava_callback - INFO - Received callback for node 69d624ad4b6493d8616634c2 INFO: 172.20.0.1:44614 - "POST /node/69d624ad4b6493d8616634c2 HTTP/1.0" 202 Accepted 2026-04-08 10:50:41,735 - lava_callback - INFO - Starting async job submit for node 69d624ad4b6493d8616634c2 2026-04-08 10:50:41,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:50:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d624ad4b6493d8616634c2 HTTP/1.1" 200 None 2026-04-08 10:50:41,776 - lava_callback - DEBUG - Processing node 69d624ad4b6493d8616634c2: baseline-nfs-arm64 2026-04-08 10:50:42,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:50:42,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:50:42,416 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d624ad4b6493d8616634c2/log.txt.gz 2026-04-08 10:50:42,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:50:42,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:50:42,892 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d624ad4b6493d8616634c2/lava-logs.yaml 2026-04-08 10:50:42,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:50:43,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:50:43,086 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d624ad4b6493d8616634c2/lava_callback.json.gz 2026-04-08 10:50:43,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:50:43,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d624ad4b6493d8616634c2 HTTP/1.1" 200 None 2026-04-08 10:50:43,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:50:43,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:50:43,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:50:43,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d624ad4b6493d8616634c2 HTTP/1.1" 200 None 2026-04-08 10:50:43,248 - lava_callback - INFO - Completed processing callback for node 69d624ad4b6493d8616634c2 INFO: 172.20.0.1:44630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42338 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:51:06,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:51:06,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57812 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-08 10:52:16,434 - lava_callback - INFO - Received callback for node 69d62eac4b6493d861663d8d INFO: 172.20.0.1:57824 - "POST /node/69d62eac4b6493d861663d8d HTTP/1.0" 202 Accepted 2026-04-08 10:52:16,442 - lava_callback - INFO - Starting async job submit for node 69d62eac4b6493d861663d8d 2026-04-08 10:52:16,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:52:16,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62eac4b6493d861663d8d HTTP/1.1" 200 None 2026-04-08 10:52:16,502 - lava_callback - DEBUG - Processing node 69d62eac4b6493d861663d8d: baseline-nfs-arm64 INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:52:17,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:52:17,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:52:17,730 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d62eac4b6493d861663d8d/log.txt.gz 2026-04-08 10:52:18,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:52:18,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:52:18,515 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d62eac4b6493d861663d8d/lava-logs.yaml 2026-04-08 10:52:18,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 10:52:18,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 10:52:18,733 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d62eac4b6493d861663d8d/lava_callback.json.gz DEBUG: Setup failed for 69d62eac4b6493d861663d8d. Transitting job node result: incomplete -> fail 2026-04-08 10:52:18,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:52:18,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d62eac4b6493d861663d8d HTTP/1.1" 200 None 2026-04-08 10:52:18,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:52:18,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 10:52:18,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:52:18,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d62eac4b6493d861663d8d HTTP/1.1" 200 None 2026-04-08 10:52:19,008 - lava_callback - INFO - Completed processing callback for node 69d62eac4b6493d861663d8d INFO: 172.20.0.1:60932 - "GET / HTTP/1.0" 200 OK 2026-04-08 10:52:36,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 10:52:36,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36504 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:05:10,260 - lava_callback - INFO - Received callback for node 69d632864b6493d861663fe9 INFO: 172.20.0.1:44690 - "POST /node/69d632864b6493d861663fe9 HTTP/1.0" 202 Accepted 2026-04-08 11:05:10,266 - lava_callback - INFO - Starting async job submit for node 69d632864b6493d861663fe9 2026-04-08 11:05:10,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:05:10,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d632864b6493d861663fe9 HTTP/1.1" 200 None 2026-04-08 11:05:10,312 - lava_callback - DEBUG - Processing node 69d632864b6493d861663fe9: baseline-arm64 2026-04-08 11:05:10,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:05:10,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:05:10,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d632864b6493d861663fe9/log.txt.gz 2026-04-08 11:05:11,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:05:11,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:05:11,412 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d632864b6493d861663fe9/lava-logs.yaml 2026-04-08 11:05:11,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:05:11,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:05:11,584 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d632864b6493d861663fe9/lava_callback.json.gz 2026-04-08 11:05:11,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:05:11,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d632864b6493d861663fe9 HTTP/1.1" 200 None 2026-04-08 11:05:11,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:05:11,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 11:05:11,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:05:11,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d632864b6493d861663fe9 HTTP/1.1" 200 None 2026-04-08 11:05:11,720 - lava_callback - INFO - Completed processing callback for node 69d632864b6493d861663fe9 INFO: 172.20.0.1:44692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47700 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:05:36,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:05:36,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2026-04-08 11:06:51,008 - lava_callback - INFO - Received callback for node 69d632c24b6493d861664012 INFO: 172.20.0.1:39884 - "POST /node/69d632c24b6493d861664012 HTTP/1.0" 202 Accepted 2026-04-08 11:06:51,015 - lava_callback - INFO - Starting async job submit for node 69d632c24b6493d861664012 2026-04-08 11:06:51,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:06:51,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d632c24b6493d861664012 HTTP/1.1" 200 None 2026-04-08 11:06:51,087 - lava_callback - DEBUG - Processing node 69d632c24b6493d861664012: baseline-arm 2026-04-08 11:06:51,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:06:51,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:06:51,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d632c24b6493d861664012/log.txt.gz 2026-04-08 11:06:52,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:06:52,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:06:52,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d632c24b6493d861664012/lava-logs.yaml 2026-04-08 11:06:52,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:06:52,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:06:52,625 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d632c24b6493d861664012/lava_callback.json.gz DEBUG: Setup failed for 69d632c24b6493d861664012. Transitting job node result: incomplete -> fail 2026-04-08 11:06:52,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:06:52,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d632c24b6493d861664012 HTTP/1.1" 200 None 2026-04-08 11:06:52,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:06:52,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f0e4b6493d861661fa0 HTTP/1.1" 200 None 2026-04-08 11:06:52,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:06:52,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d632c24b6493d861664012 HTTP/1.1" 200 None 2026-04-08 11:06:52,916 - lava_callback - INFO - Completed processing callback for node 69d632c24b6493d861664012 INFO: 172.20.0.1:39886 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:07:06,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:07:06,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:11:02,702 - lava_callback - INFO - Received callback for node 69d6330d4b6493d861664065 INFO: 172.20.0.1:45590 - "POST /node/69d6330d4b6493d861664065 HTTP/1.0" 202 Accepted 2026-04-08 11:11:02,709 - lava_callback - INFO - Starting async job submit for node 69d6330d4b6493d861664065 2026-04-08 11:11:02,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:11:02,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6330d4b6493d861664065 HTTP/1.1" 200 None 2026-04-08 11:11:02,753 - lava_callback - DEBUG - Processing node 69d6330d4b6493d861664065: baseline-nfs-arm64 2026-04-08 11:11:03,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:11:03,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:11:03,403 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6330d4b6493d861664065/log.txt.gz 2026-04-08 11:11:03,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:11:03,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:11:03,902 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6330d4b6493d861664065/lava-logs.yaml 2026-04-08 11:11:03,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:11:04,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:11:04,108 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6330d4b6493d861664065/lava_callback.json.gz 2026-04-08 11:11:04,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:11:04,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6330d4b6493d861664065 HTTP/1.1" 200 None 2026-04-08 11:11:04,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:11:04,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 11:11:04,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:11:04,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6330d4b6493d861664065 HTTP/1.1" 200 None 2026-04-08 11:11:04,299 - lava_callback - INFO - Completed processing callback for node 69d6330d4b6493d861664065 2026-04-08 11:11:06,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:11:06,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35630 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:12:37,761 - lava_callback - INFO - Received callback for node 69d6336b4b6493d8616640ad INFO: 172.20.0.1:35638 - "POST /node/69d6336b4b6493d8616640ad HTTP/1.0" 202 Accepted 2026-04-08 11:12:37,769 - lava_callback - INFO - Starting async job submit for node 69d6336b4b6493d8616640ad 2026-04-08 11:12:37,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:12:37,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6336b4b6493d8616640ad HTTP/1.1" 200 None 2026-04-08 11:12:37,830 - lava_callback - DEBUG - Processing node 69d6336b4b6493d8616640ad: baseline-nfs-arm64 2026-04-08 11:12:38,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:12:39,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:12:39,092 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6336b4b6493d8616640ad/log.txt.gz 2026-04-08 11:12:39,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:12:39,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:12:39,887 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6336b4b6493d8616640ad/lava-logs.yaml 2026-04-08 11:12:39,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:12:40,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:12:40,090 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d6336b4b6493d8616640ad/lava_callback.json.gz DEBUG: Setup failed for 69d6336b4b6493d8616640ad. Transitting job node result: incomplete -> fail 2026-04-08 11:12:40,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:12:40,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6336b4b6493d8616640ad HTTP/1.1" 200 None 2026-04-08 11:12:40,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:12:40,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 11:12:40,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:12:40,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6336b4b6493d8616640ad HTTP/1.1" 200 None 2026-04-08 11:12:40,347 - lava_callback - INFO - Completed processing callback for node 69d6336b4b6493d8616640ad INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:13:06,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:13:06,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56674 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:15:23,017 - lava_callback - INFO - Received callback for node 69d6366d4b6493d8616640e2 INFO: 172.20.0.1:42794 - "POST /node/69d6366d4b6493d8616640e2 HTTP/1.0" 202 Accepted 2026-04-08 11:15:23,024 - lava_callback - INFO - Starting async job submit for node 69d6366d4b6493d8616640e2 2026-04-08 11:15:23,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:15:23,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6366d4b6493d8616640e2 HTTP/1.1" 200 None 2026-04-08 11:15:23,075 - lava_callback - DEBUG - Processing node 69d6366d4b6493d8616640e2: baseline-arm64 2026-04-08 11:15:23,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:15:23,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:15:23,743 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6366d4b6493d8616640e2/log.txt.gz 2026-04-08 11:15:23,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:15:24,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:15:24,190 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6366d4b6493d8616640e2/lava-logs.yaml 2026-04-08 11:15:24,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:15:24,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:15:24,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d6366d4b6493d8616640e2/lava_callback.json.gz 2026-04-08 11:15:24,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:15:24,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d6366d4b6493d8616640e2 HTTP/1.1" 200 None 2026-04-08 11:15:24,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:15:24,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 11:15:24,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:15:24,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d6366d4b6493d8616640e2 HTTP/1.1" 200 None 2026-04-08 11:15:24,510 - lava_callback - INFO - Completed processing callback for node 69d6366d4b6493d8616640e2 INFO: 172.20.0.1:42800 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:15:36,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:15:36,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET / HTTP/1.0" 200 OK 2026-04-08 11:22:02,273 - lava_callback - INFO - Received callback for node 69d637d24b6493d861664118 INFO: 172.20.0.1:38088 - "POST /node/69d637d24b6493d861664118 HTTP/1.0" 202 Accepted 2026-04-08 11:22:02,279 - lava_callback - INFO - Starting async job submit for node 69d637d24b6493d861664118 2026-04-08 11:22:02,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:22:02,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d637d24b6493d861664118 HTTP/1.1" 200 None 2026-04-08 11:22:02,322 - lava_callback - DEBUG - Processing node 69d637d24b6493d861664118: baseline-nfs-arm64 2026-04-08 11:22:02,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:22:02,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:22:02,982 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d637d24b6493d861664118/log.txt.gz 2026-04-08 11:22:03,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:22:03,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:22:03,491 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d637d24b6493d861664118/lava-logs.yaml 2026-04-08 11:22:03,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-08 11:22:03,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-08 11:22:03,706 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d637d24b6493d861664118/lava_callback.json.gz 2026-04-08 11:22:03,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:22:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d637d24b6493d861664118 HTTP/1.1" 200 None 2026-04-08 11:22:03,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:22:03,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d60f1d4b6493d861661faf HTTP/1.1" 200 None 2026-04-08 11:22:03,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:22:03,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d637d24b6493d861664118 HTTP/1.1" 200 None 2026-04-08 11:22:03,846 - lava_callback - INFO - Completed processing callback for node 69d637d24b6493d861664118 2026-04-08 11:22:06,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-08 11:22:06,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]