INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47058 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:33:58,138 - lava_callback - INFO - Received callback for node 69c953fbf2866272ea51dfe7 2026-03-29 16:33:58,146 - lava_callback - INFO - Starting async job submit for node 69c953fbf2866272ea51dfe7 INFO: 172.20.0.1:47066 - "POST /node/69c953fbf2866272ea51dfe7 HTTP/1.0" 202 Accepted 2026-03-29 16:33:58,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:33:58,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c953fbf2866272ea51dfe7 HTTP/1.1" 200 None 2026-03-29 16:33:58,190 - lava_callback - DEBUG - Processing node 69c953fbf2866272ea51dfe7: baseline-arm 2026-03-29 16:33:58,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:33:58,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:33:58,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c953fbf2866272ea51dfe7/log.txt.gz 2026-03-29 16:33:58,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:33:58,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:33:58,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c953fbf2866272ea51dfe7/lava-logs.yaml 2026-03-29 16:33:58,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:33:59,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:33:59,138 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c953fbf2866272ea51dfe7/lava_callback.json.gz 2026-03-29 16:33:59,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:33:59,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c953fbf2866272ea51dfe7 HTTP/1.1" 200 None 2026-03-29 16:33:59,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:33:59,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950aef2866272ea51b8ca HTTP/1.1" 200 None 2026-03-29 16:33:59,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:33:59,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c953fbf2866272ea51dfe7 HTTP/1.1" 200 None 2026-03-29 16:33:59,245 - lava_callback - INFO - Completed processing callback for node 69c953fbf2866272ea51dfe7 INFO: 172.20.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58660 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:34:29,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:34:29,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53192 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 16:40:15,985 - lava_callback - INFO - Received callback for node 69c6b2e302375b4506df89e1 INFO: 172.20.0.1:53196 - "POST /node/69c6b2e302375b4506df89e1 HTTP/1.0" 202 Accepted 2026-03-29 16:40:15,991 - lava_callback - INFO - Starting async job submit for node 69c6b2e302375b4506df89e1 2026-03-29 16:40:15,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:16,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b2e302375b4506df89e1 HTTP/1.1" 200 None 2026-03-29 16:40:16,011 - lava_callback - DEBUG - Processing node 69c6b2e302375b4506df89e1: baseline-arm-baylibre 2026-03-29 16:40:16,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b2e302375b4506df89e1 2026-03-29 16:40:16,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:16,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b2e302375b4506df89e1 HTTP/1.1" 200 None 2026-03-29 16:40:16,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:16,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:40:16,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:16,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b2e302375b4506df89e1 HTTP/1.1" 200 None 2026-03-29 16:40:16,096 - lava_callback - ERROR - Error processing callback for node 69c6b2e302375b4506df89e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:40:55,944 - lava_callback - INFO - Received callback for node 69c6b30b02375b4506df89ed INFO: 172.20.0.1:48504 - "POST /node/69c6b30b02375b4506df89ed HTTP/1.0" 202 Accepted 2026-03-29 16:40:55,954 - lava_callback - INFO - Starting async job submit for node 69c6b30b02375b4506df89ed 2026-03-29 16:40:55,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:55,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b30b02375b4506df89ed HTTP/1.1" 200 None 2026-03-29 16:40:55,980 - lava_callback - DEBUG - Processing node 69c6b30b02375b4506df89ed: baseline-arm-baylibre 2026-03-29 16:40:55,981 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b30b02375b4506df89ed 2026-03-29 16:40:55,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:55,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b30b02375b4506df89ed HTTP/1.1" 200 None 2026-03-29 16:40:56,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:56,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:40:56,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:40:56,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b30b02375b4506df89ed HTTP/1.1" 200 None 2026-03-29 16:40:56,048 - lava_callback - ERROR - Error processing callback for node 69c6b30b02375b4506df89ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33880 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:42:30,842 - lava_callback - INFO - Received callback for node 69c9565cf2866272ea51e029 INFO: 172.20.0.1:55788 - "POST /node/69c9565cf2866272ea51e029 HTTP/1.0" 202 Accepted 2026-03-29 16:42:30,848 - lava_callback - INFO - Starting async job submit for node 69c9565cf2866272ea51e029 2026-03-29 16:42:30,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:30,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565cf2866272ea51e029 HTTP/1.1" 200 None 2026-03-29 16:42:30,878 - lava_callback - DEBUG - Processing node 69c9565cf2866272ea51e029: baseline-arm 2026-03-29 16:42:30,884 - lava_callback - WARNING - Failed to upload log 2026-03-29 16:42:30,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:31,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:31,009 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565cf2866272ea51e029/lava-logs.yaml 2026-03-29 16:42:31,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:31,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:31,140 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565cf2866272ea51e029/lava_callback.json.gz 2026-03-29 16:42:31,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:31,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565cf2866272ea51e029 HTTP/1.1" 200 None 2026-03-29 16:42:31,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:31,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:42:31,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:31,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9565cf2866272ea51e029 HTTP/1.1" 200 None 2026-03-29 16:42:31,274 - lava_callback - INFO - Completed processing callback for node 69c9565cf2866272ea51e029 INFO: 172.20.0.1:55792 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:42:42,714 - lava_callback - INFO - Received callback for node 69c95660f2866272ea51e035 INFO: 172.20.0.1:37412 - "POST /node/69c95660f2866272ea51e035 HTTP/1.0" 202 Accepted 2026-03-29 16:42:42,766 - lava_callback - INFO - Starting async job submit for node 69c95660f2866272ea51e035 2026-03-29 16:42:42,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:42,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95660f2866272ea51e035 HTTP/1.1" 200 None 2026-03-29 16:42:42,840 - lava_callback - DEBUG - Processing node 69c95660f2866272ea51e035: baseline-arm 2026-03-29 16:42:42,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:43,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:43,139 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c95660f2866272ea51e035/log.txt.gz 2026-03-29 16:42:43,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:43,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:43,375 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c95660f2866272ea51e035/lava-logs.yaml 2026-03-29 16:42:43,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:43,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:43,499 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c95660f2866272ea51e035/lava_callback.json.gz 2026-03-29 16:42:43,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:43,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95660f2866272ea51e035 HTTP/1.1" 200 None 2026-03-29 16:42:43,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:43,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:42:43,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:43,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95660f2866272ea51e035 HTTP/1.1" 200 None 2026-03-29 16:42:43,624 - lava_callback - INFO - Completed processing callback for node 69c95660f2866272ea51e035 INFO: 172.20.0.1:37416 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:42:54,081 - lava_callback - INFO - Received callback for node 69c9565df2866272ea51e02c INFO: 172.20.0.1:48928 - "POST /node/69c9565df2866272ea51e02c HTTP/1.0" 202 Accepted 2026-03-29 16:42:54,130 - lava_callback - INFO - Starting async job submit for node 69c9565df2866272ea51e02c 2026-03-29 16:42:54,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:54,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565df2866272ea51e02c HTTP/1.1" 200 None 2026-03-29 16:42:54,164 - lava_callback - DEBUG - Processing node 69c9565df2866272ea51e02c: baseline-arm 2026-03-29 16:42:54,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:54,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:54,409 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565df2866272ea51e02c/log.txt.gz 2026-03-29 16:42:54,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:54,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:54,662 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565df2866272ea51e02c/lava-logs.yaml 2026-03-29 16:42:54,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:42:54,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:42:54,779 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565df2866272ea51e02c/lava_callback.json.gz 2026-03-29 16:42:54,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:54,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565df2866272ea51e02c HTTP/1.1" 200 None 2026-03-29 16:42:54,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:54,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:42:54,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:54,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9565df2866272ea51e02c HTTP/1.1" 200 None 2026-03-29 16:42:54,866 - lava_callback - INFO - Completed processing callback for node 69c9565df2866272ea51e02c 2026-03-29 16:42:56,208 - lava_callback - INFO - Received callback for node 69c6b38302375b4506df8a07 INFO: 172.20.0.1:48938 - "POST /node/69c6b38302375b4506df8a07 HTTP/1.0" 202 Accepted 2026-03-29 16:42:56,214 - lava_callback - INFO - Starting async job submit for node 69c6b38302375b4506df8a07 2026-03-29 16:42:56,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:56,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b38302375b4506df8a07 HTTP/1.1" 200 None 2026-03-29 16:42:56,242 - lava_callback - DEBUG - Processing node 69c6b38302375b4506df8a07: baseline-arm-baylibre 2026-03-29 16:42:56,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b38302375b4506df8a07 2026-03-29 16:42:56,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:56,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b38302375b4506df8a07 HTTP/1.1" 200 None 2026-03-29 16:42:56,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:56,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:42:56,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:56,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b38302375b4506df8a07 HTTP/1.1" 200 None 2026-03-29 16:42:56,295 - lava_callback - ERROR - Error processing callback for node 69c6b38302375b4506df8a07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48948 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:42:59,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:42:59,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60706 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:43:20,574 - lava_callback - INFO - Received callback for node 69c9565ff2866272ea51e032 INFO: 172.20.0.1:42412 - "POST /node/69c9565ff2866272ea51e032 HTTP/1.0" 202 Accepted 2026-03-29 16:43:20,580 - lava_callback - INFO - Starting async job submit for node 69c9565ff2866272ea51e032 2026-03-29 16:43:20,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:20,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565ff2866272ea51e032 HTTP/1.1" 200 None 2026-03-29 16:43:20,618 - lava_callback - DEBUG - Processing node 69c9565ff2866272ea51e032: baseline-arm 2026-03-29 16:43:20,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:21,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:21,032 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565ff2866272ea51e032/log.txt.gz 2026-03-29 16:43:21,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:21,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:21,397 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565ff2866272ea51e032/lava-logs.yaml 2026-03-29 16:43:21,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:21,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:21,570 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565ff2866272ea51e032/lava_callback.json.gz 2026-03-29 16:43:21,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:21,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565ff2866272ea51e032 HTTP/1.1" 200 None 2026-03-29 16:43:21,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:21,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:43:21,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:21,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9565ff2866272ea51e032 HTTP/1.1" 200 None 2026-03-29 16:43:21,652 - lava_callback - INFO - Completed processing callback for node 69c9565ff2866272ea51e032 INFO: 172.20.0.1:42426 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:43:27,351 - lava_callback - INFO - Received callback for node 69c9565cf2866272ea51e026 INFO: 172.20.0.1:42438 - "POST /node/69c9565cf2866272ea51e026 HTTP/1.0" 202 Accepted 2026-03-29 16:43:27,358 - lava_callback - INFO - Starting async job submit for node 69c9565cf2866272ea51e026 2026-03-29 16:43:27,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:27,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565cf2866272ea51e026 HTTP/1.1" 200 None 2026-03-29 16:43:27,395 - lava_callback - DEBUG - Processing node 69c9565cf2866272ea51e026: baseline-arm 2026-03-29 16:43:27,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:27,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:27,750 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565cf2866272ea51e026/log.txt.gz 2026-03-29 16:43:27,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:28,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:28,058 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565cf2866272ea51e026/lava-logs.yaml 2026-03-29 16:43:28,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:28,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:28,208 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565cf2866272ea51e026/lava_callback.json.gz 2026-03-29 16:43:28,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:28,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565cf2866272ea51e026 HTTP/1.1" 200 None 2026-03-29 16:43:28,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:28,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:43:28,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:28,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9565cf2866272ea51e026 HTTP/1.1" 200 None 2026-03-29 16:43:28,322 - lava_callback - INFO - Completed processing callback for node 69c9565cf2866272ea51e026 2026-03-29 16:43:29,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:29,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:43:36,095 - lava_callback - INFO - Received callback for node 69c6b3ab02375b4506df8a16 INFO: 172.20.0.1:54106 - "POST /node/69c6b3ab02375b4506df8a16 HTTP/1.0" 202 Accepted 2026-03-29 16:43:36,100 - lava_callback - INFO - Starting async job submit for node 69c6b3ab02375b4506df8a16 2026-03-29 16:43:36,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:36,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b3ab02375b4506df8a16 HTTP/1.1" 200 None 2026-03-29 16:43:36,116 - lava_callback - DEBUG - Processing node 69c6b3ab02375b4506df8a16: baseline-arm-baylibre 2026-03-29 16:43:36,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b3ab02375b4506df8a16 2026-03-29 16:43:36,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:36,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b3ab02375b4506df8a16 HTTP/1.1" 200 None 2026-03-29 16:43:36,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:36,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:43:36,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:36,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b3ab02375b4506df8a16 HTTP/1.1" 200 None 2026-03-29 16:43:36,188 - lava_callback - ERROR - Error processing callback for node 69c6b3ab02375b4506df8a16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54114 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:43:38,449 - lava_callback - INFO - Received callback for node 69c9566ef2866272ea51e05b INFO: 172.20.0.1:54124 - "POST /node/69c9566ef2866272ea51e05b HTTP/1.0" 202 Accepted 2026-03-29 16:43:38,456 - lava_callback - INFO - Starting async job submit for node 69c9566ef2866272ea51e05b 2026-03-29 16:43:38,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:38,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566ef2866272ea51e05b HTTP/1.1" 200 None 2026-03-29 16:43:38,501 - lava_callback - DEBUG - Processing node 69c9566ef2866272ea51e05b: baseline-arm-broonie 2026-03-29 16:43:38,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:38,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:38,879 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566ef2866272ea51e05b/log.txt.gz 2026-03-29 16:43:38,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:39,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:39,219 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566ef2866272ea51e05b/lava-logs.yaml 2026-03-29 16:43:39,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:39,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:39,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566ef2866272ea51e05b/lava_callback.json.gz 2026-03-29 16:43:39,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:39,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566ef2866272ea51e05b HTTP/1.1" 200 None 2026-03-29 16:43:39,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:39,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:43:39,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:39,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9566ef2866272ea51e05b HTTP/1.1" 200 None 2026-03-29 16:43:39,468 - lava_callback - INFO - Completed processing callback for node 69c9566ef2866272ea51e05b 2026-03-29 16:43:39,939 - lava_callback - INFO - Received callback for node 69c9566ff2866272ea51e05e INFO: 172.20.0.1:48976 - "POST /node/69c9566ff2866272ea51e05e HTTP/1.0" 202 Accepted 2026-03-29 16:43:39,946 - lava_callback - INFO - Starting async job submit for node 69c9566ff2866272ea51e05e 2026-03-29 16:43:39,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:39,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566ff2866272ea51e05e HTTP/1.1" 200 None 2026-03-29 16:43:39,985 - lava_callback - DEBUG - Processing node 69c9566ff2866272ea51e05e: baseline-arm-broonie 2026-03-29 16:43:40,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:40,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:40,354 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566ff2866272ea51e05e/log.txt.gz 2026-03-29 16:43:40,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:40,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:40,665 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566ff2866272ea51e05e/lava-logs.yaml 2026-03-29 16:43:40,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:40,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:40,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566ff2866272ea51e05e/lava_callback.json.gz 2026-03-29 16:43:40,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:40,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566ff2866272ea51e05e HTTP/1.1" 200 None 2026-03-29 16:43:40,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:40,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:43:40,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:40,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9566ff2866272ea51e05e HTTP/1.1" 200 None 2026-03-29 16:43:40,916 - lava_callback - INFO - Completed processing callback for node 69c9566ff2866272ea51e05e INFO: 172.20.0.1:48982 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:43:54,053 - lava_callback - INFO - Received callback for node 69c95682f2866272ea51e0b3 INFO: 172.20.0.1:51102 - "POST /node/69c95682f2866272ea51e0b3 HTTP/1.0" 202 Accepted 2026-03-29 16:43:54,102 - lava_callback - INFO - Starting async job submit for node 69c95682f2866272ea51e0b3 2026-03-29 16:43:54,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:54,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95682f2866272ea51e0b3 HTTP/1.1" 200 None 2026-03-29 16:43:54,138 - lava_callback - DEBUG - Processing node 69c95682f2866272ea51e0b3: baseline-arm-pengutronix 2026-03-29 16:43:54,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:54,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:54,360 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95682f2866272ea51e0b3/log.txt.gz 2026-03-29 16:43:54,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:54,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:54,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95682f2866272ea51e0b3/lava-logs.yaml 2026-03-29 16:43:54,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:54,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:54,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95682f2866272ea51e0b3/lava_callback.json.gz 2026-03-29 16:43:54,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:54,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95682f2866272ea51e0b3 HTTP/1.1" 200 None 2026-03-29 16:43:54,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:54,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:43:54,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:54,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95682f2866272ea51e0b3 HTTP/1.1" 200 None 2026-03-29 16:43:54,787 - lava_callback - INFO - Completed processing callback for node 69c95682f2866272ea51e0b3 INFO: 172.20.0.1:51116 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:43:58,343 - lava_callback - INFO - Received callback for node 69c9567ff2866272ea51e0aa INFO: 172.20.0.1:51126 - "POST /node/69c9567ff2866272ea51e0aa HTTP/1.0" 202 Accepted 2026-03-29 16:43:58,349 - lava_callback - INFO - Starting async job submit for node 69c9567ff2866272ea51e0aa 2026-03-29 16:43:58,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:58,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9567ff2866272ea51e0aa HTTP/1.1" 200 None 2026-03-29 16:43:58,383 - lava_callback - DEBUG - Processing node 69c9567ff2866272ea51e0aa: baseline-arm-pengutronix 2026-03-29 16:43:58,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:58,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:58,642 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c9567ff2866272ea51e0aa/log.txt.gz 2026-03-29 16:43:58,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:58,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:58,951 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c9567ff2866272ea51e0aa/lava-logs.yaml 2026-03-29 16:43:58,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:59,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:59,101 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c9567ff2866272ea51e0aa/lava_callback.json.gz 2026-03-29 16:43:59,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:59,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9567ff2866272ea51e0aa HTTP/1.1" 200 None 2026-03-29 16:43:59,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:59,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:43:59,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:59,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9567ff2866272ea51e0aa HTTP/1.1" 200 None 2026-03-29 16:43:59,187 - lava_callback - INFO - Completed processing callback for node 69c9567ff2866272ea51e0aa 2026-03-29 16:43:59,204 - lava_callback - INFO - Received callback for node 69c95675f2866272ea51e087 INFO: 172.20.0.1:51138 - "POST /node/69c95675f2866272ea51e087 HTTP/1.0" 202 Accepted 2026-03-29 16:43:59,232 - lava_callback - INFO - Starting async job submit for node 69c95675f2866272ea51e087 2026-03-29 16:43:59,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:59,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95675f2866272ea51e087 HTTP/1.1" 200 None 2026-03-29 16:43:59,290 - lava_callback - DEBUG - Processing node 69c95675f2866272ea51e087: baseline-arm-clabbe 2026-03-29 16:43:59,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:43:59,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 16:43:59,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:59,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:43:59,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95675f2866272ea51e087/log.txt.gz 2026-03-29 16:43:59,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:43:59,923 - lava_callback - INFO - Received callback for node 69c95680f2866272ea51e0ad 2026-03-29 16:43:59,965 - lava_callback - INFO - Starting async job submit for node 69c95680f2866272ea51e0ad INFO: 172.20.0.1:33452 - "POST /node/69c95680f2866272ea51e0ad HTTP/1.0" 202 Accepted 2026-03-29 16:43:59,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:00,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95680f2866272ea51e0ad HTTP/1.1" 200 None 2026-03-29 16:44:00,019 - lava_callback - DEBUG - Processing node 69c95680f2866272ea51e0ad: baseline-arm-pengutronix 2026-03-29 16:44:00,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:00,161 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95675f2866272ea51e087/lava-logs.yaml 2026-03-29 16:44:00,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:00,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:00,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:00,345 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95680f2866272ea51e0ad/log.txt.gz 2026-03-29 16:44:00,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:00,414 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95675f2866272ea51e087/lava_callback.json.gz 2026-03-29 16:44:00,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:00,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:00,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95675f2866272ea51e087 HTTP/1.1" 200 None 2026-03-29 16:44:00,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:00,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:44:00,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:00,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95675f2866272ea51e087 HTTP/1.1" 200 None 2026-03-29 16:44:00,531 - lava_callback - INFO - Completed processing callback for node 69c95675f2866272ea51e087 2026-03-29 16:44:00,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:00,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95680f2866272ea51e0ad/lava-logs.yaml 2026-03-29 16:44:00,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:00,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:00,832 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95680f2866272ea51e0ad/lava_callback.json.gz 2026-03-29 16:44:00,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:00,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95680f2866272ea51e0ad HTTP/1.1" 200 None 2026-03-29 16:44:00,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:00,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:44:00,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:00,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95680f2866272ea51e0ad HTTP/1.1" 200 None 2026-03-29 16:44:00,948 - lava_callback - INFO - Completed processing callback for node 69c95680f2866272ea51e0ad INFO: 172.20.0.1:33458 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:44:07,644 - lava_callback - INFO - Received callback for node 69c95671f2866272ea51e078 INFO: 172.20.0.1:33474 - "POST /node/69c95671f2866272ea51e078 HTTP/1.0" 202 Accepted 2026-03-29 16:44:07,650 - lava_callback - INFO - Starting async job submit for node 69c95671f2866272ea51e078 2026-03-29 16:44:07,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:07,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95671f2866272ea51e078 HTTP/1.1" 200 None 2026-03-29 16:44:07,693 - lava_callback - DEBUG - Processing node 69c95671f2866272ea51e078: baseline-arm-broonie 2026-03-29 16:44:07,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:08,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:08,061 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c95671f2866272ea51e078/log.txt.gz 2026-03-29 16:44:08,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:08,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:08,421 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c95671f2866272ea51e078/lava-logs.yaml 2026-03-29 16:44:08,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:08,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:08,572 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c95671f2866272ea51e078/lava_callback.json.gz 2026-03-29 16:44:08,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:08,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95671f2866272ea51e078 HTTP/1.1" 200 None 2026-03-29 16:44:08,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:08,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:44:08,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:08,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95671f2866272ea51e078 HTTP/1.1" 200 None 2026-03-29 16:44:08,660 - lava_callback - INFO - Completed processing callback for node 69c95671f2866272ea51e078 INFO: 172.20.0.1:53262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53274 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 16:44:16,264 - lava_callback - INFO - Received callback for node 69c6b3cf02375b4506df8a2a INFO: 172.20.0.1:53278 - "POST /node/69c6b3cf02375b4506df8a2a HTTP/1.0" 202 Accepted 2026-03-29 16:44:16,273 - lava_callback - INFO - Starting async job submit for node 69c6b3cf02375b4506df8a2a 2026-03-29 16:44:16,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:16,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b3cf02375b4506df8a2a HTTP/1.1" 200 None 2026-03-29 16:44:16,292 - lava_callback - DEBUG - Processing node 69c6b3cf02375b4506df8a2a: baseline-arm-baylibre 2026-03-29 16:44:16,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b3cf02375b4506df8a2a 2026-03-29 16:44:16,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:16,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b3cf02375b4506df8a2a HTTP/1.1" 200 None 2026-03-29 16:44:16,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:16,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:44:16,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:16,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b3cf02375b4506df8a2a HTTP/1.1" 200 None 2026-03-29 16:44:16,357 - lava_callback - ERROR - Error processing callback for node 69c6b3cf02375b4506df8a2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:44:29,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:29,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60902 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:44:51,930 - lava_callback - INFO - Received callback for node 69c9569af2866272ea51e0f8 INFO: 172.20.0.1:49306 - "POST /node/69c9569af2866272ea51e0f8 HTTP/1.0" 202 Accepted 2026-03-29 16:44:51,937 - lava_callback - INFO - Starting async job submit for node 69c9569af2866272ea51e0f8 2026-03-29 16:44:51,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:51,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569af2866272ea51e0f8 HTTP/1.1" 200 None 2026-03-29 16:44:51,976 - lava_callback - DEBUG - Processing node 69c9569af2866272ea51e0f8: ltp-dio 2026-03-29 16:44:52,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:52,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:52,581 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c9569af2866272ea51e0f8/log.txt.gz 2026-03-29 16:44:52,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:53,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:53,021 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c9569af2866272ea51e0f8/lava-logs.yaml 2026-03-29 16:44:53,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:44:53,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:44:53,191 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c9569af2866272ea51e0f8/lava_callback.json.gz 2026-03-29 16:44:53,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:53,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569af2866272ea51e0f8 HTTP/1.1" 200 None 2026-03-29 16:44:53,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:53,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:44:53,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:53,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9569af2866272ea51e0f8 HTTP/1.1" 200 None 2026-03-29 16:44:53,288 - lava_callback - INFO - Completed processing callback for node 69c9569af2866272ea51e0f8 2026-03-29 16:44:56,177 - lava_callback - INFO - Received callback for node 69c6b3f502375b4506df8a47 INFO: 172.20.0.1:49320 - "POST /node/69c6b3f502375b4506df8a47 HTTP/1.0" 202 Accepted 2026-03-29 16:44:56,183 - lava_callback - INFO - Starting async job submit for node 69c6b3f502375b4506df8a47 2026-03-29 16:44:56,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:56,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b3f502375b4506df8a47 HTTP/1.1" 200 None 2026-03-29 16:44:56,200 - lava_callback - DEBUG - Processing node 69c6b3f502375b4506df8a47: baseline-arm-baylibre 2026-03-29 16:44:56,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b3f502375b4506df8a47 2026-03-29 16:44:56,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:56,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b3f502375b4506df8a47 HTTP/1.1" 200 None 2026-03-29 16:44:56,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:56,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:44:56,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:56,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b3f502375b4506df8a47 HTTP/1.1" 200 None 2026-03-29 16:44:56,268 - lava_callback - ERROR - Error processing callback for node 69c6b3f502375b4506df8a47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:44:59,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:44:59,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:45:10,466 - lava_callback - INFO - Received callback for node 69c956fdf2866272ea51e18c INFO: 172.20.0.1:56128 - "POST /node/69c956fdf2866272ea51e18c HTTP/1.0" 202 Accepted 2026-03-29 16:45:10,472 - lava_callback - INFO - Starting async job submit for node 69c956fdf2866272ea51e18c 2026-03-29 16:45:10,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:10,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956fdf2866272ea51e18c HTTP/1.1" 200 None 2026-03-29 16:45:10,490 - lava_callback - DEBUG - Processing node 69c956fdf2866272ea51e18c: baseline-arm 2026-03-29 16:45:10,495 - lava_callback - WARNING - Failed to upload log 2026-03-29 16:45:10,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:45:10,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:45:10,655 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c956fdf2866272ea51e18c/lava-logs.yaml 2026-03-29 16:45:10,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56142 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:45:10,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:45:10,815 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c956fdf2866272ea51e18c/lava_callback.json.gz 2026-03-29 16:45:10,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:10,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956fdf2866272ea51e18c HTTP/1.1" 200 None 2026-03-29 16:45:10,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:10,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:45:10,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:10,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956fdf2866272ea51e18c HTTP/1.1" 200 None 2026-03-29 16:45:10,890 - lava_callback - INFO - Completed processing callback for node 69c956fdf2866272ea51e18c INFO: 172.20.0.1:56148 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 16:45:16,197 - lava_callback - INFO - Received callback for node 69c6b41a02375b4506df8a53 INFO: 172.20.0.1:56150 - "POST /node/69c6b41a02375b4506df8a53 HTTP/1.0" 202 Accepted 2026-03-29 16:45:16,202 - lava_callback - INFO - Starting async job submit for node 69c6b41a02375b4506df8a53 2026-03-29 16:45:16,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:16,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b41a02375b4506df8a53 HTTP/1.1" 200 None 2026-03-29 16:45:16,218 - lava_callback - DEBUG - Processing node 69c6b41a02375b4506df8a53: baseline-arm-baylibre 2026-03-29 16:45:16,219 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b41a02375b4506df8a53 2026-03-29 16:45:16,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:16,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b41a02375b4506df8a53 HTTP/1.1" 200 None 2026-03-29 16:45:16,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:16,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:45:16,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:16,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b41a02375b4506df8a53 HTTP/1.1" 200 None 2026-03-29 16:45:16,286 - lava_callback - ERROR - Error processing callback for node 69c6b41a02375b4506df8a53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56152 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:45:26,207 - lava_callback - INFO - Received callback for node 69c95676f2866272ea51e08b INFO: 172.20.0.1:38424 - "POST /node/69c95676f2866272ea51e08b HTTP/1.0" 202 Accepted 2026-03-29 16:45:26,213 - lava_callback - INFO - Starting async job submit for node 69c95676f2866272ea51e08b 2026-03-29 16:45:26,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:26,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95676f2866272ea51e08b HTTP/1.1" 200 None 2026-03-29 16:45:26,257 - lava_callback - DEBUG - Processing node 69c95676f2866272ea51e08b: baseline-arm-clabbe 2026-03-29 16:45:26,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:38428 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:45:26,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:45:26,586 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95676f2866272ea51e08b/log.txt.gz 2026-03-29 16:45:26,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:45:26,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:45:26,911 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95676f2866272ea51e08b/lava-logs.yaml 2026-03-29 16:45:26,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:45:27,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:45:27,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95676f2866272ea51e08b/lava_callback.json.gz 2026-03-29 16:45:27,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:27,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95676f2866272ea51e08b HTTP/1.1" 200 None 2026-03-29 16:45:27,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:27,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:45:27,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:27,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95676f2866272ea51e08b HTTP/1.1" 200 None 2026-03-29 16:45:27,133 - lava_callback - INFO - Completed processing callback for node 69c95676f2866272ea51e08b 2026-03-29 16:45:29,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:29,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46760 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:45:56,236 - lava_callback - INFO - Received callback for node 69c6b43e02375b4506df8a5e INFO: 172.20.0.1:45836 - "POST /node/69c6b43e02375b4506df8a5e HTTP/1.0" 202 Accepted 2026-03-29 16:45:56,242 - lava_callback - INFO - Starting async job submit for node 69c6b43e02375b4506df8a5e 2026-03-29 16:45:56,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:56,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b43e02375b4506df8a5e HTTP/1.1" 200 None 2026-03-29 16:45:56,257 - lava_callback - DEBUG - Processing node 69c6b43e02375b4506df8a5e: baseline-arm-baylibre 2026-03-29 16:45:56,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b43e02375b4506df8a5e 2026-03-29 16:45:56,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:56,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b43e02375b4506df8a5e HTTP/1.1" 200 None 2026-03-29 16:45:56,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:56,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:45:56,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:45:56,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b43e02375b4506df8a5e HTTP/1.1" 200 None 2026-03-29 16:45:56,312 - lava_callback - ERROR - Error processing callback for node 69c6b43e02375b4506df8a5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36034 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:46:36,283 - lava_callback - INFO - Received callback for node 69c6b46302375b4506df8a6c INFO: 172.20.0.1:46034 - "POST /node/69c6b46302375b4506df8a6c HTTP/1.0" 202 Accepted 2026-03-29 16:46:36,292 - lava_callback - INFO - Starting async job submit for node 69c6b46302375b4506df8a6c 2026-03-29 16:46:36,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:46:36,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b46302375b4506df8a6c HTTP/1.1" 200 None 2026-03-29 16:46:36,335 - lava_callback - DEBUG - Processing node 69c6b46302375b4506df8a6c: baseline-arm-baylibre 2026-03-29 16:46:36,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b46302375b4506df8a6c 2026-03-29 16:46:36,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:46:36,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b46302375b4506df8a6c HTTP/1.1" 200 None 2026-03-29 16:46:36,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:46:36,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:46:36,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:46:36,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b46302375b4506df8a6c HTTP/1.1" 200 None 2026-03-29 16:46:36,458 - lava_callback - ERROR - Error processing callback for node 69c6b46302375b4506df8a6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41548 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:47:05,337 - lava_callback - INFO - Received callback for node 69c95679f2866272ea51e099 INFO: 172.20.0.1:51726 - "POST /node/69c95679f2866272ea51e099 HTTP/1.0" 202 Accepted 2026-03-29 16:47:05,347 - lava_callback - INFO - Starting async job submit for node 69c95679f2866272ea51e099 2026-03-29 16:47:05,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:05,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95679f2866272ea51e099 HTTP/1.1" 200 None 2026-03-29 16:47:05,427 - lava_callback - DEBUG - Processing node 69c95679f2866272ea51e099: baseline-arm-clabbe 2026-03-29 16:47:05,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:06,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:06,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95679f2866272ea51e099/log.txt.gz 2026-03-29 16:47:06,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51730 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:47:06,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:06,899 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95679f2866272ea51e099/lava-logs.yaml 2026-03-29 16:47:06,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:07,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:07,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c95679f2866272ea51e099/lava_callback.json.gz 2026-03-29 16:47:07,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:07,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95679f2866272ea51e099 HTTP/1.1" 200 None 2026-03-29 16:47:07,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:07,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:47:07,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:07,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95679f2866272ea51e099 HTTP/1.1" 200 None 2026-03-29 16:47:07,449 - lava_callback - INFO - Completed processing callback for node 69c95679f2866272ea51e099 2026-03-29 16:47:07,613 - lava_callback - INFO - Received callback for node 69c95693f2866272ea51e0e4 INFO: 172.20.0.1:51742 - "POST /node/69c95693f2866272ea51e0e4 HTTP/1.0" 202 Accepted 2026-03-29 16:47:07,625 - lava_callback - INFO - Starting async job submit for node 69c95693f2866272ea51e0e4 2026-03-29 16:47:07,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:07,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95693f2866272ea51e0e4 HTTP/1.1" 200 None 2026-03-29 16:47:07,753 - lava_callback - DEBUG - Processing node 69c95693f2866272ea51e0e4: ltp-cap_bounds 2026-03-29 16:47:08,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:08,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:08,763 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c95693f2866272ea51e0e4/log.txt.gz 2026-03-29 16:47:09,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:09,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:09,748 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c95693f2866272ea51e0e4/lava-logs.yaml 2026-03-29 16:47:09,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:10,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:10,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c95693f2866272ea51e0e4/lava_callback.json.gz 2026-03-29 16:47:10,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:10,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95693f2866272ea51e0e4 HTTP/1.1" 200 None 2026-03-29 16:47:10,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:10,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:47:10,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:10,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95693f2866272ea51e0e4 HTTP/1.1" 200 None 2026-03-29 16:47:10,289 - lava_callback - INFO - Completed processing callback for node 69c95693f2866272ea51e0e4 INFO: 172.20.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55662 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 16:47:16,346 - lava_callback - INFO - Received callback for node 69c6b48802375b4506df8a77 INFO: 172.20.0.1:55672 - "POST /node/69c6b48802375b4506df8a77 HTTP/1.0" 202 Accepted 2026-03-29 16:47:16,355 - lava_callback - INFO - Starting async job submit for node 69c6b48802375b4506df8a77 2026-03-29 16:47:16,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:16,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b48802375b4506df8a77 HTTP/1.1" 200 None 2026-03-29 16:47:16,386 - lava_callback - DEBUG - Processing node 69c6b48802375b4506df8a77: baseline-arm-baylibre 2026-03-29 16:47:16,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b48802375b4506df8a77 2026-03-29 16:47:16,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:16,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b48802375b4506df8a77 HTTP/1.1" 200 None 2026-03-29 16:47:16,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:16,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:47:16,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:16,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b48802375b4506df8a77 HTTP/1.1" 200 None 2026-03-29 16:47:16,495 - lava_callback - ERROR - Error processing callback for node 69c6b48802375b4506df8a77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55676 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:47:18,844 - lava_callback - INFO - Received callback for node 69c95698f2866272ea51e0f5 INFO: 172.20.0.1:55682 - "POST /node/69c95698f2866272ea51e0f5 HTTP/1.0" 202 Accepted 2026-03-29 16:47:18,854 - lava_callback - INFO - Starting async job submit for node 69c95698f2866272ea51e0f5 2026-03-29 16:47:18,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:18,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95698f2866272ea51e0f5 HTTP/1.1" 200 None 2026-03-29 16:47:18,935 - lava_callback - DEBUG - Processing node 69c95698f2866272ea51e0f5: ltp-cve 2026-03-29 16:47:19,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:20,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:20,088 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c95698f2866272ea51e0f5/log.txt.gz 2026-03-29 16:47:20,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:20,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:20,991 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c95698f2866272ea51e0f5/lava-logs.yaml 2026-03-29 16:47:21,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:21,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:21,418 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c95698f2866272ea51e0f5/lava_callback.json.gz 2026-03-29 16:47:21,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:21,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95698f2866272ea51e0f5 HTTP/1.1" 200 None 2026-03-29 16:47:21,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:21,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:47:21,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:21,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95698f2866272ea51e0f5 HTTP/1.1" 200 None 2026-03-29 16:47:21,593 - lava_callback - INFO - Completed processing callback for node 69c95698f2866272ea51e0f5 INFO: 172.20.0.1:60054 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:47:27,702 - lava_callback - INFO - Received callback for node 69c9566df2866272ea51e053 INFO: 172.20.0.1:60066 - "POST /node/69c9566df2866272ea51e053 HTTP/1.0" 202 Accepted 2026-03-29 16:47:27,711 - lava_callback - INFO - Starting async job submit for node 69c9566df2866272ea51e053 2026-03-29 16:47:27,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:27,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566df2866272ea51e053 HTTP/1.1" 200 None 2026-03-29 16:47:27,784 - lava_callback - DEBUG - Processing node 69c9566df2866272ea51e053: baseline-arm-broonie 2026-03-29 16:47:28,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:28,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:28,427 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566df2866272ea51e053/log.txt.gz 2026-03-29 16:47:28,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:29,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:29,086 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566df2866272ea51e053/lava-logs.yaml 2026-03-29 16:47:29,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:29,330 - lava_callback - INFO - Received callback for node 69c95696f2866272ea51e0f2 INFO: 172.20.0.1:60070 - "POST /node/69c95696f2866272ea51e0f2 HTTP/1.0" 202 Accepted 2026-03-29 16:47:29,341 - lava_callback - INFO - Starting async job submit for node 69c95696f2866272ea51e0f2 2026-03-29 16:47:29,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:29,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95696f2866272ea51e0f2 HTTP/1.1" 200 None 2026-03-29 16:47:29,414 - lava_callback - DEBUG - Processing node 69c95696f2866272ea51e0f2: ltp-controllers 2026-03-29 16:47:29,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:29,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:47:29,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:29,632 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c9566df2866272ea51e053/lava_callback.json.gz 2026-03-29 16:47:29,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:29,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566df2866272ea51e053 HTTP/1.1" 200 None 2026-03-29 16:47:29,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:29,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:47:29,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:29,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9566df2866272ea51e053 HTTP/1.1" 200 None 2026-03-29 16:47:29,925 - lava_callback - INFO - Completed processing callback for node 69c9566df2866272ea51e053 2026-03-29 16:47:30,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:30,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:30,902 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c95696f2866272ea51e0f2/log.txt.gz 2026-03-29 16:47:31,011 - lava_callback - INFO - Received callback for node 69c956a6f2866272ea51e110 INFO: 172.20.0.1:48210 - "POST /node/69c956a6f2866272ea51e110 HTTP/1.0" 202 Accepted 2026-03-29 16:47:31,056 - lava_callback - INFO - Starting async job submit for node 69c956a6f2866272ea51e110 2026-03-29 16:47:31,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:31,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a6f2866272ea51e110 HTTP/1.1" 200 None 2026-03-29 16:47:31,226 - lava_callback - DEBUG - Processing node 69c956a6f2866272ea51e110: ltp-syscalls 2026-03-29 16:47:31,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:32,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:32,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c95696f2866272ea51e0f2/lava-logs.yaml 2026-03-29 16:47:32,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:32,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:33,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:33,068 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c95696f2866272ea51e0f2/lava_callback.json.gz 2026-03-29 16:47:33,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:33,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95696f2866272ea51e0f2 HTTP/1.1" 200 None 2026-03-29 16:47:33,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:33,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:47:33,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:33,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95696f2866272ea51e0f2 HTTP/1.1" 200 None 2026-03-29 16:47:33,217 - lava_callback - INFO - Completed processing callback for node 69c95696f2866272ea51e0f2 2026-03-29 16:47:33,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:33,279 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c956a6f2866272ea51e110/log.txt.gz 2026-03-29 16:47:34,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:34,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:34,569 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c956a6f2866272ea51e110/lava-logs.yaml 2026-03-29 16:47:34,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:35,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:35,019 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c956a6f2866272ea51e110/lava_callback.json.gz 2026-03-29 16:47:35,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:35,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a6f2866272ea51e110 HTTP/1.1" 200 None 2026-03-29 16:47:35,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:35,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:47:35,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:35,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956a6f2866272ea51e110 HTTP/1.1" 200 None 2026-03-29 16:47:35,119 - lava_callback - INFO - Completed processing callback for node 69c956a6f2866272ea51e110 INFO: 172.20.0.1:48212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45948 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:47:48,022 - lava_callback - INFO - Received callback for node 69c95694f2866272ea51e0ef INFO: 172.20.0.1:45956 - "POST /node/69c95694f2866272ea51e0ef HTTP/1.0" 202 Accepted 2026-03-29 16:47:48,033 - lava_callback - INFO - Starting async job submit for node 69c95694f2866272ea51e0ef 2026-03-29 16:47:48,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:48,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95694f2866272ea51e0ef HTTP/1.1" 200 None 2026-03-29 16:47:48,106 - lava_callback - DEBUG - Processing node 69c95694f2866272ea51e0ef: ltp-containers 2026-03-29 16:47:48,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:49,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:49,153 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c95694f2866272ea51e0ef/log.txt.gz 2026-03-29 16:47:49,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:50,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:50,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c95694f2866272ea51e0ef/lava-logs.yaml 2026-03-29 16:47:50,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:47:50,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:47:50,484 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c95694f2866272ea51e0ef/lava_callback.json.gz 2026-03-29 16:47:50,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:50,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95694f2866272ea51e0ef HTTP/1.1" 200 None 2026-03-29 16:47:50,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:50,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:47:50,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:50,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95694f2866272ea51e0ef HTTP/1.1" 200 None 2026-03-29 16:47:50,643 - lava_callback - INFO - Completed processing callback for node 69c95694f2866272ea51e0ef 2026-03-29 16:47:56,328 - lava_callback - INFO - Received callback for node 69c6b4ac02375b4506df8a84 INFO: 172.20.0.1:54622 - "POST /node/69c6b4ac02375b4506df8a84 HTTP/1.0" 202 Accepted 2026-03-29 16:47:56,334 - lava_callback - INFO - Starting async job submit for node 69c6b4ac02375b4506df8a84 2026-03-29 16:47:56,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:56,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b4ac02375b4506df8a84 HTTP/1.1" 200 None 2026-03-29 16:47:56,350 - lava_callback - DEBUG - Processing node 69c6b4ac02375b4506df8a84: baseline-arm-baylibre 2026-03-29 16:47:56,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b4ac02375b4506df8a84 2026-03-29 16:47:56,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:56,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b4ac02375b4506df8a84 HTTP/1.1" 200 None 2026-03-29 16:47:56,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:56,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:47:56,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:56,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b4ac02375b4506df8a84 HTTP/1.1" 200 None 2026-03-29 16:47:56,410 - lava_callback - ERROR - Error processing callback for node 69c6b4ac02375b4506df8a84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54626 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:47:59,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:47:59,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39470 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:48:24,634 - lava_callback - INFO - Received callback for node 69c6b4d402375b4506df8aa1 INFO: 172.20.0.1:42422 - "POST /node/69c6b4d402375b4506df8aa1 HTTP/1.0" 202 Accepted 2026-03-29 16:48:24,639 - lava_callback - INFO - Starting async job submit for node 69c6b4d402375b4506df8aa1 2026-03-29 16:48:24,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:24,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b4d402375b4506df8aa1 HTTP/1.1" 200 None 2026-03-29 16:48:24,655 - lava_callback - DEBUG - Processing node 69c6b4d402375b4506df8aa1: baseline-arm-clabbe 2026-03-29 16:48:24,655 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b4d402375b4506df8aa1 2026-03-29 16:48:24,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:24,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b4d402375b4506df8aa1 HTTP/1.1" 200 None 2026-03-29 16:48:24,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:24,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:48:24,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:24,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b4d402375b4506df8aa1 HTTP/1.1" 200 None 2026-03-29 16:48:24,731 - lava_callback - ERROR - Error processing callback for node 69c6b4d402375b4506df8aa1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50580 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:48:48,348 - lava_callback - INFO - Received callback for node 69c9567af2866272ea51e09c INFO: 172.20.0.1:50596 - "POST /node/69c9567af2866272ea51e09c HTTP/1.0" 202 Accepted 2026-03-29 16:48:48,355 - lava_callback - INFO - Starting async job submit for node 69c9567af2866272ea51e09c 2026-03-29 16:48:48,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:48,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9567af2866272ea51e09c HTTP/1.1" 200 None 2026-03-29 16:48:48,393 - lava_callback - DEBUG - Processing node 69c9567af2866272ea51e09c: baseline-arm-clabbe 2026-03-29 16:48:48,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:48:48,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:48:48,759 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c9567af2866272ea51e09c/log.txt.gz 2026-03-29 16:48:48,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:48:49,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:48:49,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c9567af2866272ea51e09c/lava-logs.yaml 2026-03-29 16:48:49,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:48:49,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:48:49,253 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c9567af2866272ea51e09c/lava_callback.json.gz 2026-03-29 16:48:49,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:49,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9567af2866272ea51e09c HTTP/1.1" 200 None 2026-03-29 16:48:49,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:49,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:48:49,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:49,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9567af2866272ea51e09c HTTP/1.1" 200 None 2026-03-29 16:48:49,341 - lava_callback - INFO - Completed processing callback for node 69c9567af2866272ea51e09c INFO: 172.20.0.1:41840 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:48:59,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:48:59,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:49:04,618 - lava_callback - INFO - Received callback for node 69c6b4f902375b4506df8aaf INFO: 172.20.0.1:38564 - "POST /node/69c6b4f902375b4506df8aaf HTTP/1.0" 202 Accepted 2026-03-29 16:49:04,624 - lava_callback - INFO - Starting async job submit for node 69c6b4f902375b4506df8aaf 2026-03-29 16:49:04,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:04,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b4f902375b4506df8aaf HTTP/1.1" 200 None 2026-03-29 16:49:04,640 - lava_callback - DEBUG - Processing node 69c6b4f902375b4506df8aaf: baseline-arm-clabbe 2026-03-29 16:49:04,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b4f902375b4506df8aaf 2026-03-29 16:49:04,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:04,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b4f902375b4506df8aaf HTTP/1.1" 200 None 2026-03-29 16:49:04,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:04,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:49:04,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:04,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b4f902375b4506df8aaf HTTP/1.1" 200 None 2026-03-29 16:49:04,697 - lava_callback - ERROR - Error processing callback for node 69c6b4f902375b4506df8aaf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36042 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:49:44,667 - lava_callback - INFO - Received callback for node 69c6b51f02375b4506df8ab8 INFO: 172.20.0.1:50592 - "POST /node/69c6b51f02375b4506df8ab8 HTTP/1.0" 202 Accepted 2026-03-29 16:49:44,673 - lava_callback - INFO - Starting async job submit for node 69c6b51f02375b4506df8ab8 2026-03-29 16:49:44,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:44,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b51f02375b4506df8ab8 HTTP/1.1" 200 None 2026-03-29 16:49:44,698 - lava_callback - DEBUG - Processing node 69c6b51f02375b4506df8ab8: baseline-arm-clabbe 2026-03-29 16:49:44,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b51f02375b4506df8ab8 2026-03-29 16:49:44,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:44,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b51f02375b4506df8ab8 HTTP/1.1" 200 None 2026-03-29 16:49:44,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:44,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:49:44,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:49:44,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b51f02375b4506df8ab8 HTTP/1.1" 200 None 2026-03-29 16:49:44,766 - lava_callback - ERROR - Error processing callback for node 69c6b51f02375b4506df8ab8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33400 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:50:24,686 - lava_callback - INFO - Received callback for node 69c6b54302375b4506df8ac2 INFO: 172.20.0.1:33880 - "POST /node/69c6b54302375b4506df8ac2 HTTP/1.0" 202 Accepted 2026-03-29 16:50:24,695 - lava_callback - INFO - Starting async job submit for node 69c6b54302375b4506df8ac2 2026-03-29 16:50:24,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:50:24,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b54302375b4506df8ac2 HTTP/1.1" 200 None 2026-03-29 16:50:24,731 - lava_callback - DEBUG - Processing node 69c6b54302375b4506df8ac2: baseline-arm-clabbe 2026-03-29 16:50:24,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b54302375b4506df8ac2 2026-03-29 16:50:24,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:50:24,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b54302375b4506df8ac2 HTTP/1.1" 200 None 2026-03-29 16:50:24,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:50:24,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:50:24,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:50:24,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b54302375b4506df8ac2 HTTP/1.1" 200 None 2026-03-29 16:50:24,822 - lava_callback - ERROR - Error processing callback for node 69c6b54302375b4506df8ac2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49146 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:51:04,962 - lava_callback - INFO - Received callback for node 69c6b56802375b4506df8acc INFO: 172.20.0.1:39482 - "POST /node/69c6b56802375b4506df8acc HTTP/1.0" 202 Accepted 2026-03-29 16:51:04,971 - lava_callback - INFO - Starting async job submit for node 69c6b56802375b4506df8acc 2026-03-29 16:51:04,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:05,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b56802375b4506df8acc HTTP/1.1" 200 None 2026-03-29 16:51:05,005 - lava_callback - DEBUG - Processing node 69c6b56802375b4506df8acc: baseline-arm-clabbe 2026-03-29 16:51:05,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b56802375b4506df8acc 2026-03-29 16:51:05,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:05,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b56802375b4506df8acc HTTP/1.1" 200 None 2026-03-29 16:51:05,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:05,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 16:51:05,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:05,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b56802375b4506df8acc HTTP/1.1" 200 None 2026-03-29 16:51:05,120 - lava_callback - ERROR - Error processing callback for node 69c6b56802375b4506df8acc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39492 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:51:07,828 - lava_callback - INFO - Received callback for node 69c9569bf2866272ea51e0fb INFO: 172.20.0.1:39496 - "POST /node/69c9569bf2866272ea51e0fb HTTP/1.0" 202 Accepted 2026-03-29 16:51:07,839 - lava_callback - INFO - Starting async job submit for node 69c9569bf2866272ea51e0fb 2026-03-29 16:51:07,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:07,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569bf2866272ea51e0fb HTTP/1.1" 200 None 2026-03-29 16:51:07,928 - lava_callback - DEBUG - Processing node 69c9569bf2866272ea51e0fb: ltp-fs_bind 2026-03-29 16:51:08,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:09,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:09,239 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c9569bf2866272ea51e0fb/log.txt.gz 2026-03-29 16:51:09,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:10,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:10,572 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c9569bf2866272ea51e0fb/lava-logs.yaml 2026-03-29 16:51:10,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:51:11,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:11,336 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c9569bf2866272ea51e0fb/lava_callback.json.gz 2026-03-29 16:51:11,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:11,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569bf2866272ea51e0fb HTTP/1.1" 200 None 2026-03-29 16:51:11,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:11,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:51:11,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:11,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9569bf2866272ea51e0fb HTTP/1.1" 200 None 2026-03-29 16:51:11,500 - lava_callback - INFO - Completed processing callback for node 69c9569bf2866272ea51e0fb INFO: 172.20.0.1:47514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:51:18,667 - lava_callback - INFO - Received callback for node 69c9569ef2866272ea51e101 INFO: 172.20.0.1:47534 - "POST /node/69c9569ef2866272ea51e101 HTTP/1.0" 202 Accepted 2026-03-29 16:51:18,677 - lava_callback - INFO - Starting async job submit for node 69c9569ef2866272ea51e101 2026-03-29 16:51:18,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:18,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569ef2866272ea51e101 HTTP/1.1" 200 None 2026-03-29 16:51:18,785 - lava_callback - DEBUG - Processing node 69c9569ef2866272ea51e101: ltp-input 2026-03-29 16:51:19,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:19,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:19,875 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c9569ef2866272ea51e101/log.txt.gz 2026-03-29 16:51:20,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:21,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:21,236 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c9569ef2866272ea51e101/lava-logs.yaml 2026-03-29 16:51:21,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:21,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:21,650 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c9569ef2866272ea51e101/lava_callback.json.gz 2026-03-29 16:51:21,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:21,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569ef2866272ea51e101 HTTP/1.1" 200 None 2026-03-29 16:51:21,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:21,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:51:21,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:21,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9569ef2866272ea51e101 HTTP/1.1" 200 None 2026-03-29 16:51:21,788 - lava_callback - INFO - Completed processing callback for node 69c9569ef2866272ea51e101 INFO: 172.20.0.1:57546 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:51:28,031 - lava_callback - INFO - Received callback for node 69c9569df2866272ea51e0fe INFO: 172.20.0.1:57552 - "POST /node/69c9569df2866272ea51e0fe HTTP/1.0" 202 Accepted 2026-03-29 16:51:28,042 - lava_callback - INFO - Starting async job submit for node 69c9569df2866272ea51e0fe 2026-03-29 16:51:28,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:28,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569df2866272ea51e0fe HTTP/1.1" 200 None 2026-03-29 16:51:28,140 - lava_callback - DEBUG - Processing node 69c9569df2866272ea51e0fe: ltp-fsx 2026-03-29 16:51:28,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:29,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:29,177 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c9569df2866272ea51e0fe/log.txt.gz 2026-03-29 16:51:29,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:29,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:29,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:51:30,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:30,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c9569df2866272ea51e0fe/lava-logs.yaml 2026-03-29 16:51:30,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:30,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:30,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c9569df2866272ea51e0fe/lava_callback.json.gz 2026-03-29 16:51:30,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:30,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9569df2866272ea51e0fe HTTP/1.1" 200 None 2026-03-29 16:51:30,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:31,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:51:31,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:31,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9569df2866272ea51e0fe HTTP/1.1" 200 None 2026-03-29 16:51:31,074 - lava_callback - INFO - Completed processing callback for node 69c9569df2866272ea51e0fe INFO: 172.20.0.1:47688 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:51:36,680 - lava_callback - INFO - Received callback for node 69c6b58b02375b4506df8ada INFO: 172.20.0.1:47702 - "POST /node/69c6b58b02375b4506df8ada HTTP/1.0" 202 Accepted 2026-03-29 16:51:36,689 - lava_callback - INFO - Starting async job submit for node 69c6b58b02375b4506df8ada 2026-03-29 16:51:36,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:36,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b58b02375b4506df8ada HTTP/1.1" 200 None 2026-03-29 16:51:36,715 - lava_callback - DEBUG - Processing node 69c6b58b02375b4506df8ada: baseline-arm-baylibre 2026-03-29 16:51:36,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b58b02375b4506df8ada 2026-03-29 16:51:36,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:36,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b58b02375b4506df8ada HTTP/1.1" 200 None 2026-03-29 16:51:36,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:36,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:51:36,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:36,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b58b02375b4506df8ada HTTP/1.1" 200 None 2026-03-29 16:51:36,813 - lava_callback - ERROR - Error processing callback for node 69c6b58b02375b4506df8ada: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:51:38,919 - lava_callback - INFO - Received callback for node 69c953fdf2866272ea51dfea INFO: 172.20.0.1:47714 - "POST /node/69c953fdf2866272ea51dfea HTTP/1.0" 202 Accepted 2026-03-29 16:51:38,930 - lava_callback - INFO - Starting async job submit for node 69c953fdf2866272ea51dfea 2026-03-29 16:51:39,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:39,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c953fdf2866272ea51dfea HTTP/1.1" 200 None 2026-03-29 16:51:39,061 - lava_callback - DEBUG - Processing node 69c953fdf2866272ea51dfea: ltp-smoke 2026-03-29 16:51:39,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:40,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:40,735 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c953fdf2866272ea51dfea/log.txt.gz 2026-03-29 16:51:41,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:42,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:42,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c953fdf2866272ea51dfea/lava-logs.yaml 2026-03-29 16:51:42,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:42,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:42,889 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c953fdf2866272ea51dfea/lava_callback.json.gz 2026-03-29 16:51:43,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:43,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c953fdf2866272ea51dfea HTTP/1.1" 200 None 2026-03-29 16:51:43,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:43,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950aef2866272ea51b8ca HTTP/1.1" 200 None 2026-03-29 16:51:43,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:43,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c953fdf2866272ea51dfea HTTP/1.1" 200 None 2026-03-29 16:51:43,375 - lava_callback - INFO - Completed processing callback for node 69c953fdf2866272ea51dfea INFO: 172.20.0.1:59358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58512 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:51:58,438 - lava_callback - INFO - Received callback for node 69c9589ff2866272ea51e280 INFO: 172.20.0.1:58526 - "POST /node/69c9589ff2866272ea51e280 HTTP/1.0" 202 Accepted 2026-03-29 16:51:58,446 - lava_callback - INFO - Starting async job submit for node 69c9589ff2866272ea51e280 2026-03-29 16:51:58,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:58,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9589ff2866272ea51e280 HTTP/1.1" 200 None 2026-03-29 16:51:58,494 - lava_callback - DEBUG - Processing node 69c9589ff2866272ea51e280: baseline-arm 2026-03-29 16:51:58,502 - lava_callback - WARNING - Failed to upload log 2026-03-29 16:51:58,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:59,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:51:59,402 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c9589ff2866272ea51e280/lava-logs.yaml 2026-03-29 16:51:59,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:51:59,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:51:59,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:52:00,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:00,229 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c9589ff2866272ea51e280/lava_callback.json.gz 2026-03-29 16:52:00,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:00,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9589ff2866272ea51e280 HTTP/1.1" 200 None 2026-03-29 16:52:00,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:00,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:52:00,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:00,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9589ff2866272ea51e280 HTTP/1.1" 200 None 2026-03-29 16:52:00,348 - lava_callback - INFO - Completed processing callback for node 69c9589ff2866272ea51e280 INFO: 172.20.0.1:51108 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:52:07,800 - lava_callback - INFO - Received callback for node 69c956a0f2866272ea51e104 INFO: 172.20.0.1:51120 - "POST /node/69c956a0f2866272ea51e104 HTTP/1.0" 202 Accepted 2026-03-29 16:52:07,808 - lava_callback - INFO - Starting async job submit for node 69c956a0f2866272ea51e104 2026-03-29 16:52:07,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:07,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a0f2866272ea51e104 HTTP/1.1" 200 None 2026-03-29 16:52:07,857 - lava_callback - DEBUG - Processing node 69c956a0f2866272ea51e104: ltp-ipc 2026-03-29 16:52:08,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:09,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:09,311 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c956a0f2866272ea51e104/log.txt.gz 2026-03-29 16:52:09,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:10,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:10,060 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c956a0f2866272ea51e104/lava-logs.yaml 2026-03-29 16:52:10,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:10,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:10,495 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c956a0f2866272ea51e104/lava_callback.json.gz 2026-03-29 16:52:10,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:10,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a0f2866272ea51e104 HTTP/1.1" 200 None 2026-03-29 16:52:10,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:10,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:52:10,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:10,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956a0f2866272ea51e104 HTTP/1.1" 200 None 2026-03-29 16:52:10,618 - lava_callback - INFO - Completed processing callback for node 69c956a0f2866272ea51e104 INFO: 172.20.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:52:24,784 - lava_callback - INFO - Received callback for node 69c6b5b302375b4506df8ae2 INFO: 172.20.0.1:35410 - "POST /node/69c6b5b302375b4506df8ae2 HTTP/1.0" 202 Accepted 2026-03-29 16:52:24,797 - lava_callback - INFO - Starting async job submit for node 69c6b5b302375b4506df8ae2 2026-03-29 16:52:24,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:24,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b5b302375b4506df8ae2 HTTP/1.1" 200 None 2026-03-29 16:52:24,838 - lava_callback - DEBUG - Processing node 69c6b5b302375b4506df8ae2: baseline-arm-clabbe 2026-03-29 16:52:24,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b5b302375b4506df8ae2 2026-03-29 16:52:24,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:24,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b5b302375b4506df8ae2 HTTP/1.1" 200 None 2026-03-29 16:52:24,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:24,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:52:24,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:24,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b5b302375b4506df8ae2 HTTP/1.1" 200 None 2026-03-29 16:52:24,926 - lava_callback - ERROR - Error processing callback for node 69c6b5b302375b4506df8ae2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35424 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:52:28,088 - lava_callback - INFO - Received callback for node 69c956a1f2866272ea51e107 INFO: 172.20.0.1:35426 - "POST /node/69c956a1f2866272ea51e107 HTTP/1.0" 202 Accepted 2026-03-29 16:52:28,095 - lava_callback - INFO - Starting async job submit for node 69c956a1f2866272ea51e107 2026-03-29 16:52:28,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:28,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a1f2866272ea51e107 HTTP/1.1" 200 None 2026-03-29 16:52:28,152 - lava_callback - DEBUG - Processing node 69c956a1f2866272ea51e107: ltp-pty 2026-03-29 16:52:28,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:28,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:28,941 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c956a1f2866272ea51e107/log.txt.gz 2026-03-29 16:52:29,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:29,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:29,668 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c956a1f2866272ea51e107/lava-logs.yaml 2026-03-29 16:52:29,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:29,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:29,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:52:29,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:29,828 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c956a1f2866272ea51e107/lava_callback.json.gz 2026-03-29 16:52:29,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:29,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a1f2866272ea51e107 HTTP/1.1" 200 None 2026-03-29 16:52:29,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:29,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:52:29,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:30,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956a1f2866272ea51e107 HTTP/1.1" 200 None 2026-03-29 16:52:30,017 - lava_callback - INFO - Completed processing callback for node 69c956a1f2866272ea51e107 INFO: 172.20.0.1:51578 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:52:43,535 - lava_callback - INFO - Received callback for node 69c958bcf2866272ea51e2a4 INFO: 172.20.0.1:47666 - "POST /node/69c958bcf2866272ea51e2a4 HTTP/1.0" 202 Accepted 2026-03-29 16:52:43,541 - lava_callback - INFO - Starting async job submit for node 69c958bcf2866272ea51e2a4 2026-03-29 16:52:43,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:43,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bcf2866272ea51e2a4 HTTP/1.1" 200 None 2026-03-29 16:52:43,578 - lava_callback - DEBUG - Processing node 69c958bcf2866272ea51e2a4: baseline-arm64 2026-03-29 16:52:43,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:44,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:44,212 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bcf2866272ea51e2a4/log.txt.gz 2026-03-29 16:52:44,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:44,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:44,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bcf2866272ea51e2a4/lava-logs.yaml 2026-03-29 16:52:44,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:45,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:45,070 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bcf2866272ea51e2a4/lava_callback.json.gz 2026-03-29 16:52:45,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:45,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bcf2866272ea51e2a4 HTTP/1.1" 200 None 2026-03-29 16:52:45,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:45,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:52:45,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:45,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958bcf2866272ea51e2a4 HTTP/1.1" 200 None 2026-03-29 16:52:45,156 - lava_callback - INFO - Completed processing callback for node 69c958bcf2866272ea51e2a4 INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:52:57,294 - lava_callback - INFO - Received callback for node 69c958ddf2866272ea51e30f INFO: 172.20.0.1:46108 - "POST /node/69c958ddf2866272ea51e30f HTTP/1.0" 202 Accepted 2026-03-29 16:52:57,305 - lava_callback - INFO - Starting async job submit for node 69c958ddf2866272ea51e30f 2026-03-29 16:52:57,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:57,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958ddf2866272ea51e30f HTTP/1.1" 200 None 2026-03-29 16:52:57,348 - lava_callback - DEBUG - Processing node 69c958ddf2866272ea51e30f: baseline-nfs-arm64 2026-03-29 16:52:57,349 - lava_callback - WARNING - Failed to upload log 2026-03-29 16:52:57,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:57,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:57,814 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958ddf2866272ea51e30f/lava-logs.yaml 2026-03-29 16:52:57,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:52:58,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:52:58,016 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958ddf2866272ea51e30f/lava_callback.json.gz 2026-03-29 16:52:58,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:58,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958ddf2866272ea51e30f HTTP/1.1" 200 None 2026-03-29 16:52:58,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:58,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:52:58,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:58,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958ddf2866272ea51e30f HTTP/1.1" 200 None 2026-03-29 16:52:58,113 - lava_callback - INFO - Completed processing callback for node 69c958ddf2866272ea51e30f 2026-03-29 16:52:59,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:52:59,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:53:00,569 - lava_callback - INFO - Received callback for node 69c958cbf2866272ea51e2dc INFO: 172.20.0.1:56584 - "POST /node/69c958cbf2866272ea51e2dc HTTP/1.0" 202 Accepted 2026-03-29 16:53:00,576 - lava_callback - INFO - Starting async job submit for node 69c958cbf2866272ea51e2dc 2026-03-29 16:53:00,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:00,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958cbf2866272ea51e2dc HTTP/1.1" 200 None 2026-03-29 16:53:00,610 - lava_callback - DEBUG - Processing node 69c958cbf2866272ea51e2dc: baseline-arm64 2026-03-29 16:53:00,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:01,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:01,200 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cbf2866272ea51e2dc/log.txt.gz 2026-03-29 16:53:01,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:01,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:01,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cbf2866272ea51e2dc/lava-logs.yaml 2026-03-29 16:53:01,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:01,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:01,898 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cbf2866272ea51e2dc/lava_callback.json.gz 2026-03-29 16:53:01,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:01,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958cbf2866272ea51e2dc HTTP/1.1" 200 None 2026-03-29 16:53:01,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:01,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:01,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:02,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958cbf2866272ea51e2dc HTTP/1.1" 200 None 2026-03-29 16:53:02,039 - lava_callback - INFO - Completed processing callback for node 69c958cbf2866272ea51e2dc 2026-03-29 16:53:04,759 - lava_callback - INFO - Received callback for node 69c6b5da02375b4506df8af9 INFO: 172.20.0.1:56596 - "POST /node/69c6b5da02375b4506df8af9 HTTP/1.0" 202 Accepted 2026-03-29 16:53:04,770 - lava_callback - INFO - Starting async job submit for node 69c6b5da02375b4506df8af9 2026-03-29 16:53:04,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:04,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b5da02375b4506df8af9 HTTP/1.1" 200 None 2026-03-29 16:53:04,816 - lava_callback - DEBUG - Processing node 69c6b5da02375b4506df8af9: baseline-arm-clabbe 2026-03-29 16:53:04,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b5da02375b4506df8af9 2026-03-29 16:53:04,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:04,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b5da02375b4506df8af9 HTTP/1.1" 200 None 2026-03-29 16:53:04,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:04,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:53:04,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:04,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b5da02375b4506df8af9 HTTP/1.1" 200 None 2026-03-29 16:53:04,900 - lava_callback - ERROR - Error processing callback for node 69c6b5da02375b4506df8af9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:53:24,800 - lava_callback - INFO - Received callback for node 69c6b60002375b4506df8b73 INFO: 172.20.0.1:34088 - "POST /node/69c6b60002375b4506df8b73 HTTP/1.0" 202 Accepted 2026-03-29 16:53:24,808 - lava_callback - INFO - Starting async job submit for node 69c6b60002375b4506df8b73 2026-03-29 16:53:24,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:24,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60002375b4506df8b73 HTTP/1.1" 200 None 2026-03-29 16:53:24,839 - lava_callback - DEBUG - Processing node 69c6b60002375b4506df8b73: baseline-arm-clabbe 2026-03-29 16:53:24,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b60002375b4506df8b73 2026-03-29 16:53:24,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:24,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60002375b4506df8b73 HTTP/1.1" 200 None 2026-03-29 16:53:24,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:24,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:53:24,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:24,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b60002375b4506df8b73 HTTP/1.1" 200 None 2026-03-29 16:53:24,930 - lava_callback - ERROR - Error processing callback for node 69c6b60002375b4506df8b73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:53:25,127 - lava_callback - INFO - Received callback for node 69c958c1f2866272ea51e2b0 INFO: 172.20.0.1:34098 - "POST /node/69c958c1f2866272ea51e2b0 HTTP/1.0" 202 Accepted 2026-03-29 16:53:25,172 - lava_callback - INFO - Starting async job submit for node 69c958c1f2866272ea51e2b0 2026-03-29 16:53:25,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:25,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c1f2866272ea51e2b0 HTTP/1.1" 200 None 2026-03-29 16:53:25,352 - lava_callback - DEBUG - Processing node 69c958c1f2866272ea51e2b0: baseline-arm64 2026-03-29 16:53:25,604 - lava_callback - WARNING - Failed to upload log 2026-03-29 16:53:25,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:26,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:26,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c1f2866272ea51e2b0/lava-logs.yaml 2026-03-29 16:53:26,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34112 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:53:26,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:26,546 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c1f2866272ea51e2b0/lava_callback.json.gz 2026-03-29 16:53:26,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:26,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c1f2866272ea51e2b0 HTTP/1.1" 200 None 2026-03-29 16:53:26,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:26,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:26,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:26,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c1f2866272ea51e2b0 HTTP/1.1" 200 None 2026-03-29 16:53:26,849 - lava_callback - INFO - Completed processing callback for node 69c958c1f2866272ea51e2b0 2026-03-29 16:53:29,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:29,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:53:30,298 - lava_callback - INFO - Received callback for node 69c958c2f2866272ea51e2b3 INFO: 172.20.0.1:53352 - "POST /node/69c958c2f2866272ea51e2b3 HTTP/1.0" 202 Accepted 2026-03-29 16:53:30,315 - lava_callback - INFO - Starting async job submit for node 69c958c2f2866272ea51e2b3 2026-03-29 16:53:30,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:30,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c2f2866272ea51e2b3 HTTP/1.1" 200 None 2026-03-29 16:53:30,413 - lava_callback - DEBUG - Processing node 69c958c2f2866272ea51e2b3: baseline-arm64 2026-03-29 16:53:30,642 - lava_callback - WARNING - Failed to upload log 2026-03-29 16:53:30,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:31,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:31,275 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c2f2866272ea51e2b3/lava-logs.yaml 2026-03-29 16:53:31,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:31,415 - lava_callback - INFO - Received callback for node 69c958bef2866272ea51e2aa INFO: 172.20.0.1:53364 - "POST /node/69c958bef2866272ea51e2aa HTTP/1.0" 202 Accepted 2026-03-29 16:53:31,426 - lava_callback - INFO - Starting async job submit for node 69c958bef2866272ea51e2aa 2026-03-29 16:53:31,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:31,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:31,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c2f2866272ea51e2b3/lava_callback.json.gz 2026-03-29 16:53:31,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bef2866272ea51e2aa HTTP/1.1" 200 None 2026-03-29 16:53:31,511 - lava_callback - DEBUG - Processing node 69c958bef2866272ea51e2aa: baseline-arm64 2026-03-29 16:53:31,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:31,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c2f2866272ea51e2b3 HTTP/1.1" 200 None 2026-03-29 16:53:31,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:31,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:31,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:31,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c2f2866272ea51e2b3 HTTP/1.1" 200 None 2026-03-29 16:53:32,026 - lava_callback - INFO - Completed processing callback for node 69c958c2f2866272ea51e2b3 2026-03-29 16:53:32,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:32,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:32,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bef2866272ea51e2aa/log.txt.gz 2026-03-29 16:53:33,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:33,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:33,754 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bef2866272ea51e2aa/lava-logs.yaml 2026-03-29 16:53:33,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:33,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:33,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bef2866272ea51e2aa/lava_callback.json.gz 2026-03-29 16:53:33,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:33,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bef2866272ea51e2aa HTTP/1.1" 200 None 2026-03-29 16:53:33,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:34,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:34,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:34,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958bef2866272ea51e2aa HTTP/1.1" 200 None 2026-03-29 16:53:34,116 - lava_callback - INFO - Completed processing callback for node 69c958bef2866272ea51e2aa INFO: 172.20.0.1:53378 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:53:36,692 - lava_callback - INFO - Received callback for node 69c6b60102375b4506df8b76 INFO: 172.20.0.1:53382 - "POST /node/69c6b60102375b4506df8b76 HTTP/1.0" 202 Accepted 2026-03-29 16:53:36,701 - lava_callback - INFO - Starting async job submit for node 69c6b60102375b4506df8b76 2026-03-29 16:53:36,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:36,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60102375b4506df8b76 HTTP/1.1" 200 None 2026-03-29 16:53:36,727 - lava_callback - DEBUG - Processing node 69c6b60102375b4506df8b76: baseline-arm64-baylibre 2026-03-29 16:53:36,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b60102375b4506df8b76 2026-03-29 16:53:36,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:36,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60102375b4506df8b76 HTTP/1.1" 200 None 2026-03-29 16:53:36,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:36,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:53:36,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:36,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b60102375b4506df8b76 HTTP/1.1" 200 None 2026-03-29 16:53:36,810 - lava_callback - ERROR - Error processing callback for node 69c6b60102375b4506df8b76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:53:37,389 - lava_callback - INFO - Received callback for node 69c6b60202375b4506df8b99 INFO: 172.20.0.1:53390 - "POST /node/69c6b60202375b4506df8b99 HTTP/1.0" 202 Accepted 2026-03-29 16:53:37,398 - lava_callback - INFO - Starting async job submit for node 69c6b60202375b4506df8b99 2026-03-29 16:53:37,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:37,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60202375b4506df8b99 HTTP/1.1" 200 None 2026-03-29 16:53:37,434 - lava_callback - DEBUG - Processing node 69c6b60202375b4506df8b99: baseline-arm64-baylibre 2026-03-29 16:53:37,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b60202375b4506df8b99 2026-03-29 16:53:37,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:37,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60202375b4506df8b99 HTTP/1.1" 200 None 2026-03-29 16:53:37,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:37,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:53:37,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:37,507 - lava_callback - INFO - Received callback for node 69c958c6f2866272ea51e2bd INFO: 172.20.0.1:53400 - "POST /node/69c958c6f2866272ea51e2bd HTTP/1.0" 202 Accepted 2026-03-29 16:53:37,526 - lava_callback - INFO - Starting async job submit for node 69c958c6f2866272ea51e2bd 2026-03-29 16:53:37,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b60202375b4506df8b99 HTTP/1.1" 200 None 2026-03-29 16:53:37,546 - lava_callback - ERROR - Error processing callback for node 69c6b60202375b4506df8b99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:53:37,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:37,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c6f2866272ea51e2bd HTTP/1.1" 200 None 2026-03-29 16:53:37,642 - lava_callback - DEBUG - Processing node 69c958c6f2866272ea51e2bd: baseline-arm64 2026-03-29 16:53:38,119 - lava_callback - INFO - Received callback for node 69c6b60702375b4506df8baa INFO: 172.20.0.1:53410 - "POST /node/69c6b60702375b4506df8baa HTTP/1.0" 202 Accepted 2026-03-29 16:53:38,141 - lava_callback - INFO - Starting async job submit for node 69c6b60702375b4506df8baa 2026-03-29 16:53:38,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:38,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60702375b4506df8baa HTTP/1.1" 200 None 2026-03-29 16:53:38,249 - lava_callback - DEBUG - Processing node 69c6b60702375b4506df8baa: baseline-arm64-baylibre 2026-03-29 16:53:38,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b60702375b4506df8baa 2026-03-29 16:53:38,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:38,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60702375b4506df8baa HTTP/1.1" 200 None 2026-03-29 16:53:38,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:38,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:38,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:53:38,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:38,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b60702375b4506df8baa HTTP/1.1" 200 None 2026-03-29 16:53:38,520 - lava_callback - ERROR - Error processing callback for node 69c6b60702375b4506df8baa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:53:38,803 - lava_callback - INFO - Received callback for node 69c6b5ff02375b4506df8b6d INFO: 172.20.0.1:53414 - "POST /node/69c6b5ff02375b4506df8b6d HTTP/1.0" 202 Accepted 2026-03-29 16:53:38,813 - lava_callback - INFO - Starting async job submit for node 69c6b5ff02375b4506df8b6d 2026-03-29 16:53:38,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:38,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c6f2866272ea51e2bd/log.txt.gz 2026-03-29 16:53:38,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:38,871 - lava_callback - INFO - Received callback for node 69c958ccf2866272ea51e2df 2026-03-29 16:53:38,900 - lava_callback - INFO - Starting async job submit for node 69c958ccf2866272ea51e2df INFO: 172.20.0.1:53420 - "POST /node/69c958ccf2866272ea51e2df HTTP/1.0" 202 Accepted 2026-03-29 16:53:38,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b5ff02375b4506df8b6d HTTP/1.1" 200 None 2026-03-29 16:53:38,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:38,972 - lava_callback - DEBUG - Processing node 69c6b5ff02375b4506df8b6d: baseline-arm64-baylibre 2026-03-29 16:53:38,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b5ff02375b4506df8b6d 2026-03-29 16:53:38,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:39,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b5ff02375b4506df8b6d HTTP/1.1" 200 None 2026-03-29 16:53:39,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958ccf2866272ea51e2df HTTP/1.1" 200 None 2026-03-29 16:53:39,071 - lava_callback - DEBUG - Processing node 69c958ccf2866272ea51e2df: baseline-arm64 2026-03-29 16:53:39,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:39,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:53:39,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:39,382 - lava_callback - INFO - Received callback for node 69c958c8f2866272ea51e2c0 2026-03-29 16:53:39,456 - lava_callback - INFO - Starting async job submit for node 69c958c8f2866272ea51e2c0 INFO: 172.20.0.1:53424 - "POST /node/69c958c8f2866272ea51e2c0 HTTP/1.0" 202 Accepted 2026-03-29 16:53:39,489 - lava_callback - INFO - Received callback for node 69c6b60402375b4506df8ba0 2026-03-29 16:53:39,551 - lava_callback - INFO - Starting async job submit for node 69c6b60402375b4506df8ba0 INFO: 172.20.0.1:53428 - "POST /node/69c6b60402375b4506df8ba0 HTTP/1.0" 202 Accepted 2026-03-29 16:53:39,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:39,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b5ff02375b4506df8b6d HTTP/1.1" 200 None 2026-03-29 16:53:39,618 - lava_callback - ERROR - Error processing callback for node 69c6b5ff02375b4506df8b6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:53:39,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:39,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60402375b4506df8ba0 HTTP/1.1" 200 None 2026-03-29 16:53:39,861 - lava_callback - DEBUG - Processing node 69c6b60402375b4506df8ba0: baseline-arm64-baylibre 2026-03-29 16:53:39,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b60402375b4506df8ba0 2026-03-29 16:53:39,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:39,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:39,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c8f2866272ea51e2c0 HTTP/1.1" 200 None 2026-03-29 16:53:39,965 - lava_callback - DEBUG - Processing node 69c958c8f2866272ea51e2c0: baseline-arm64 2026-03-29 16:53:39,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60402375b4506df8ba0 HTTP/1.1" 200 None 2026-03-29 16:53:40,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:40,246 - lava_callback - INFO - Received callback for node 69c6b60602375b4506df8ba7 2026-03-29 16:53:40,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None INFO: 172.20.0.1:35058 - "POST /node/69c6b60602375b4506df8ba7 HTTP/1.0" 202 Accepted 2026-03-29 16:53:40,281 - lava_callback - INFO - Starting async job submit for node 69c6b60602375b4506df8ba7 2026-03-29 16:53:40,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:40,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:40,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:40,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60602375b4506df8ba7 HTTP/1.1" 200 None 2026-03-29 16:53:40,419 - lava_callback - DEBUG - Processing node 69c6b60602375b4506df8ba7: baseline-arm64-baylibre 2026-03-29 16:53:40,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b60602375b4506df8ba7 2026-03-29 16:53:40,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:40,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b60402375b4506df8ba0 HTTP/1.1" 200 None 2026-03-29 16:53:40,432 - lava_callback - ERROR - Error processing callback for node 69c6b60402375b4506df8ba0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:53:40,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b60602375b4506df8ba7 HTTP/1.1" 200 None 2026-03-29 16:53:40,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:40,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:53:40,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:40,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b60602375b4506df8ba7 HTTP/1.1" 200 None 2026-03-29 16:53:40,909 - lava_callback - ERROR - Error processing callback for node 69c6b60602375b4506df8ba7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:53:41,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:41,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:41,479 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c6f2866272ea51e2bd/lava-logs.yaml 2026-03-29 16:53:41,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:41,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:41,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958ccf2866272ea51e2df/log.txt.gz 2026-03-29 16:53:41,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:41,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c8f2866272ea51e2c0/log.txt.gz 2026-03-29 16:53:41,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:41,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c6f2866272ea51e2bd/lava_callback.json.gz 2026-03-29 16:53:41,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:41,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:42,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c6f2866272ea51e2bd HTTP/1.1" 200 None 2026-03-29 16:53:42,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:42,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:42,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:42,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c6f2866272ea51e2bd HTTP/1.1" 200 None 2026-03-29 16:53:42,251 - lava_callback - INFO - Completed processing callback for node 69c958c6f2866272ea51e2bd 2026-03-29 16:53:42,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:42,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:42,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958ccf2866272ea51e2df/lava-logs.yaml 2026-03-29 16:53:42,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:42,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:42,814 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c8f2866272ea51e2c0/lava-logs.yaml 2026-03-29 16:53:42,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:42,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:42,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958ccf2866272ea51e2df/lava_callback.json.gz 2026-03-29 16:53:42,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:42,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958ccf2866272ea51e2df HTTP/1.1" 200 None 2026-03-29 16:53:42,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:42,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:42,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:43,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958ccf2866272ea51e2df HTTP/1.1" 200 None 2026-03-29 16:53:43,006 - lava_callback - INFO - Completed processing callback for node 69c958ccf2866272ea51e2df 2026-03-29 16:53:43,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:43,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c8f2866272ea51e2c0/lava_callback.json.gz 2026-03-29 16:53:43,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:43,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c8f2866272ea51e2c0 HTTP/1.1" 200 None 2026-03-29 16:53:43,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:43,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:43,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:43,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c8f2866272ea51e2c0 HTTP/1.1" 200 None 2026-03-29 16:53:43,618 - lava_callback - INFO - Completed processing callback for node 69c958c8f2866272ea51e2c0 2026-03-29 16:53:45,370 - lava_callback - INFO - Received callback for node 69c958d1f2866272ea51e2f0 INFO: 172.20.0.1:35072 - "POST /node/69c958d1f2866272ea51e2f0 HTTP/1.0" 202 Accepted 2026-03-29 16:53:45,380 - lava_callback - INFO - Starting async job submit for node 69c958d1f2866272ea51e2f0 2026-03-29 16:53:45,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:45,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d1f2866272ea51e2f0 HTTP/1.1" 200 None 2026-03-29 16:53:45,455 - lava_callback - DEBUG - Processing node 69c958d1f2866272ea51e2f0: baseline-arm64 2026-03-29 16:53:45,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:45,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:45,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d1f2866272ea51e2f0/log.txt.gz 2026-03-29 16:53:46,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:35076 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:53:46,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:46,598 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d1f2866272ea51e2f0/lava-logs.yaml 2026-03-29 16:53:46,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:46,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:46,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d1f2866272ea51e2f0/lava_callback.json.gz 2026-03-29 16:53:46,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:46,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d1f2866272ea51e2f0 HTTP/1.1" 200 None 2026-03-29 16:53:46,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:46,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:46,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:46,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958d1f2866272ea51e2f0 HTTP/1.1" 200 None 2026-03-29 16:53:46,894 - lava_callback - INFO - Completed processing callback for node 69c958d1f2866272ea51e2f0 2026-03-29 16:53:48,252 - lava_callback - INFO - Received callback for node 69c958c9f2866272ea51e2d9 INFO: 172.20.0.1:35088 - "POST /node/69c958c9f2866272ea51e2d9 HTTP/1.0" 202 Accepted 2026-03-29 16:53:48,262 - lava_callback - INFO - Starting async job submit for node 69c958c9f2866272ea51e2d9 2026-03-29 16:53:48,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:48,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c9f2866272ea51e2d9 HTTP/1.1" 200 None 2026-03-29 16:53:48,357 - lava_callback - DEBUG - Processing node 69c958c9f2866272ea51e2d9: baseline-arm64 2026-03-29 16:53:49,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:49,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:49,500 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c9f2866272ea51e2d9/log.txt.gz 2026-03-29 16:53:49,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:50,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:50,369 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c9f2866272ea51e2d9/lava-logs.yaml 2026-03-29 16:53:50,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:50,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:50,891 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c9f2866272ea51e2d9/lava_callback.json.gz 2026-03-29 16:53:50,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:50,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c9f2866272ea51e2d9 HTTP/1.1" 200 None 2026-03-29 16:53:50,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:50,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:50,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:50,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c9f2866272ea51e2d9 HTTP/1.1" 200 None 2026-03-29 16:53:50,991 - lava_callback - INFO - Completed processing callback for node 69c958c9f2866272ea51e2d9 2026-03-29 16:53:52,141 - lava_callback - INFO - Received callback for node 69c958c5f2866272ea51e2b9 INFO: 172.20.0.1:51316 - "POST /node/69c958c5f2866272ea51e2b9 HTTP/1.0" 202 Accepted 2026-03-29 16:53:52,148 - lava_callback - INFO - Starting async job submit for node 69c958c5f2866272ea51e2b9 2026-03-29 16:53:52,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:52,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c5f2866272ea51e2b9 HTTP/1.1" 200 None 2026-03-29 16:53:52,186 - lava_callback - DEBUG - Processing node 69c958c5f2866272ea51e2b9: baseline-arm64 2026-03-29 16:53:52,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:52,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:52,568 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c5f2866272ea51e2b9/log.txt.gz 2026-03-29 16:53:52,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:52,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:52,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c5f2866272ea51e2b9/lava-logs.yaml 2026-03-29 16:53:52,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:53,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:53,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c5f2866272ea51e2b9/lava_callback.json.gz 2026-03-29 16:53:53,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:53,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c5f2866272ea51e2b9 HTTP/1.1" 200 None 2026-03-29 16:53:53,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:53,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:53,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:53,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c5f2866272ea51e2b9 HTTP/1.1" 200 None 2026-03-29 16:53:53,235 - lava_callback - INFO - Completed processing callback for node 69c958c5f2866272ea51e2b9 2026-03-29 16:53:54,671 - lava_callback - INFO - Received callback for node 69c958bdf2866272ea51e2a7 INFO: 172.20.0.1:51326 - "POST /node/69c958bdf2866272ea51e2a7 HTTP/1.0" 202 Accepted 2026-03-29 16:53:54,678 - lava_callback - INFO - Starting async job submit for node 69c958bdf2866272ea51e2a7 2026-03-29 16:53:54,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:54,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bdf2866272ea51e2a7 HTTP/1.1" 200 None 2026-03-29 16:53:54,733 - lava_callback - DEBUG - Processing node 69c958bdf2866272ea51e2a7: baseline-arm64 2026-03-29 16:53:54,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:55,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:55,108 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bdf2866272ea51e2a7/log.txt.gz 2026-03-29 16:53:55,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:55,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:55,432 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bdf2866272ea51e2a7/lava-logs.yaml 2026-03-29 16:53:55,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:53:55,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:53:55,567 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bdf2866272ea51e2a7/lava_callback.json.gz 2026-03-29 16:53:55,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:55,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bdf2866272ea51e2a7 HTTP/1.1" 200 None 2026-03-29 16:53:55,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:55,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:53:55,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:55,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958bdf2866272ea51e2a7 HTTP/1.1" 200 None 2026-03-29 16:53:55,660 - lava_callback - INFO - Completed processing callback for node 69c958bdf2866272ea51e2a7 INFO: 172.20.0.1:51328 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:53:59,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:53:59,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 16:54:02,500 - lava_callback - INFO - Received callback for node 69c958cff2866272ea51e2ed INFO: 172.20.0.1:55720 - "POST /node/69c958cff2866272ea51e2ed HTTP/1.0" 202 Accepted 2026-03-29 16:54:02,506 - lava_callback - INFO - Starting async job submit for node 69c958cff2866272ea51e2ed 2026-03-29 16:54:02,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:02,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958cff2866272ea51e2ed HTTP/1.1" 200 None 2026-03-29 16:54:02,544 - lava_callback - DEBUG - Processing node 69c958cff2866272ea51e2ed: baseline-arm64 2026-03-29 16:54:02,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:02,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:02,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cff2866272ea51e2ed/log.txt.gz 2026-03-29 16:54:03,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:03,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:03,326 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cff2866272ea51e2ed/lava-logs.yaml 2026-03-29 16:54:03,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:03,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:03,460 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cff2866272ea51e2ed/lava_callback.json.gz 2026-03-29 16:54:03,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:03,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958cff2866272ea51e2ed HTTP/1.1" 200 None 2026-03-29 16:54:03,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:03,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:03,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:03,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958cff2866272ea51e2ed HTTP/1.1" 200 None 2026-03-29 16:54:03,547 - lava_callback - INFO - Completed processing callback for node 69c958cff2866272ea51e2ed 2026-03-29 16:54:03,849 - lava_callback - INFO - Received callback for node 69c958d2f2866272ea51e2f3 INFO: 172.20.0.1:55730 - "POST /node/69c958d2f2866272ea51e2f3 HTTP/1.0" 202 Accepted 2026-03-29 16:54:03,856 - lava_callback - INFO - Starting async job submit for node 69c958d2f2866272ea51e2f3 2026-03-29 16:54:03,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:03,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d2f2866272ea51e2f3 HTTP/1.1" 200 None 2026-03-29 16:54:03,897 - lava_callback - DEBUG - Processing node 69c958d2f2866272ea51e2f3: baseline-arm64 2026-03-29 16:54:04,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:04,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:04,417 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d2f2866272ea51e2f3/log.txt.gz 2026-03-29 16:54:04,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:04,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:04,805 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d2f2866272ea51e2f3/lava-logs.yaml 2026-03-29 16:54:04,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:04,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:04,944 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d2f2866272ea51e2f3/lava_callback.json.gz 2026-03-29 16:54:04,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:04,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d2f2866272ea51e2f3 HTTP/1.1" 200 None 2026-03-29 16:54:04,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:04,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:04,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:05,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958d2f2866272ea51e2f3 HTTP/1.1" 200 None 2026-03-29 16:54:05,026 - lava_callback - INFO - Completed processing callback for node 69c958d2f2866272ea51e2f3 INFO: 172.20.0.1:55734 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:54:09,395 - lava_callback - INFO - Received callback for node 69c6b61702375b4506df8bd0 INFO: 172.20.0.1:55736 - "POST /node/69c6b61702375b4506df8bd0 HTTP/1.0" 202 Accepted 2026-03-29 16:54:09,401 - lava_callback - INFO - Starting async job submit for node 69c6b61702375b4506df8bd0 2026-03-29 16:54:09,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:09,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61702375b4506df8bd0 HTTP/1.1" 200 None 2026-03-29 16:54:09,417 - lava_callback - DEBUG - Processing node 69c6b61702375b4506df8bd0: baseline-arm64-clabbe 2026-03-29 16:54:09,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b61702375b4506df8bd0 2026-03-29 16:54:09,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:09,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61702375b4506df8bd0 HTTP/1.1" 200 None 2026-03-29 16:54:09,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:09,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:54:09,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:09,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b61702375b4506df8bd0 HTTP/1.1" 200 None 2026-03-29 16:54:09,481 - lava_callback - ERROR - Error processing callback for node 69c6b61702375b4506df8bd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:54:10,501 - lava_callback - INFO - Received callback for node 69c6b61902375b4506df8bd5 INFO: 172.20.0.1:45192 - "POST /node/69c6b61902375b4506df8bd5 HTTP/1.0" 202 Accepted 2026-03-29 16:54:10,506 - lava_callback - INFO - Starting async job submit for node 69c6b61902375b4506df8bd5 2026-03-29 16:54:10,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:10,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61902375b4506df8bd5 HTTP/1.1" 200 None 2026-03-29 16:54:10,523 - lava_callback - DEBUG - Processing node 69c6b61902375b4506df8bd5: baseline-arm64-clabbe 2026-03-29 16:54:10,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b61902375b4506df8bd5 2026-03-29 16:54:10,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:10,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61902375b4506df8bd5 HTTP/1.1" 200 None 2026-03-29 16:54:10,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:10,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:54:10,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:10,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b61902375b4506df8bd5 HTTP/1.1" 200 None 2026-03-29 16:54:10,579 - lava_callback - ERROR - Error processing callback for node 69c6b61902375b4506df8bd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:54:11,575 - lava_callback - INFO - Received callback for node 69c6b61902375b4506df8bd8 INFO: 172.20.0.1:45214 - "POST /node/69c6b61902375b4506df8bd8 HTTP/1.0" 202 Accepted 2026-03-29 16:54:11,580 - lava_callback - INFO - Starting async job submit for node 69c6b61902375b4506df8bd8 2026-03-29 16:54:11,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:11,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61902375b4506df8bd8 HTTP/1.1" 200 None 2026-03-29 16:54:11,596 - lava_callback - DEBUG - Processing node 69c6b61902375b4506df8bd8: baseline-arm64-clabbe 2026-03-29 16:54:11,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b61902375b4506df8bd8 2026-03-29 16:54:11,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:11,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61902375b4506df8bd8 HTTP/1.1" 200 None 2026-03-29 16:54:11,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:11,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:54:11,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:11,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b61902375b4506df8bd8 HTTP/1.1" 200 None 2026-03-29 16:54:11,653 - lava_callback - ERROR - Error processing callback for node 69c6b61902375b4506df8bd8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:54:12,237 - lava_callback - INFO - Received callback for node 69c6b61a02375b4506df8bdb INFO: 172.20.0.1:45216 - "POST /node/69c6b61a02375b4506df8bdb HTTP/1.0" 202 Accepted 2026-03-29 16:54:12,242 - lava_callback - INFO - Starting async job submit for node 69c6b61a02375b4506df8bdb 2026-03-29 16:54:12,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61a02375b4506df8bdb HTTP/1.1" 200 None 2026-03-29 16:54:12,258 - lava_callback - DEBUG - Processing node 69c6b61a02375b4506df8bdb: baseline-arm64-clabbe 2026-03-29 16:54:12,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b61a02375b4506df8bdb 2026-03-29 16:54:12,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61a02375b4506df8bdb HTTP/1.1" 200 None 2026-03-29 16:54:12,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:54:12,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b61a02375b4506df8bdb HTTP/1.1" 200 None 2026-03-29 16:54:12,329 - lava_callback - ERROR - Error processing callback for node 69c6b61a02375b4506df8bdb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:54:12,903 - lava_callback - INFO - Received callback for node 69c6b61c02375b4506df8be3 INFO: 172.20.0.1:45230 - "POST /node/69c6b61c02375b4506df8be3 HTTP/1.0" 202 Accepted 2026-03-29 16:54:12,908 - lava_callback - INFO - Starting async job submit for node 69c6b61c02375b4506df8be3 2026-03-29 16:54:12,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61c02375b4506df8be3 HTTP/1.1" 200 None 2026-03-29 16:54:12,926 - lava_callback - DEBUG - Processing node 69c6b61c02375b4506df8be3: baseline-arm64-clabbe 2026-03-29 16:54:12,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b61c02375b4506df8be3 2026-03-29 16:54:12,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61c02375b4506df8be3 HTTP/1.1" 200 None 2026-03-29 16:54:12,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:54:12,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:12,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b61c02375b4506df8be3 HTTP/1.1" 200 None 2026-03-29 16:54:12,990 - lava_callback - ERROR - Error processing callback for node 69c6b61c02375b4506df8be3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:54:13,539 - lava_callback - INFO - Received callback for node 69c958bff2866272ea51e2ad INFO: 172.20.0.1:45238 - "POST /node/69c958bff2866272ea51e2ad HTTP/1.0" 202 Accepted 2026-03-29 16:54:13,545 - lava_callback - INFO - Starting async job submit for node 69c958bff2866272ea51e2ad 2026-03-29 16:54:13,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:13,577 - lava_callback - INFO - Received callback for node 69c6b61d02375b4506df8be6 INFO: 172.20.0.1:45250 - "POST /node/69c6b61d02375b4506df8be6 HTTP/1.0" 202 Accepted 2026-03-29 16:54:13,583 - lava_callback - INFO - Starting async job submit for node 69c6b61d02375b4506df8be6 2026-03-29 16:54:13,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:13,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bff2866272ea51e2ad HTTP/1.1" 200 None 2026-03-29 16:54:13,593 - lava_callback - DEBUG - Processing node 69c958bff2866272ea51e2ad: baseline-arm64 2026-03-29 16:54:13,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61d02375b4506df8be6 HTTP/1.1" 200 None 2026-03-29 16:54:13,648 - lava_callback - DEBUG - Processing node 69c6b61d02375b4506df8be6: baseline-arm64-clabbe 2026-03-29 16:54:13,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b61d02375b4506df8be6 2026-03-29 16:54:13,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:13,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b61d02375b4506df8be6 HTTP/1.1" 200 None 2026-03-29 16:54:13,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:13,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 16:54:13,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:13,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:13,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b61d02375b4506df8be6 HTTP/1.1" 200 None 2026-03-29 16:54:13,986 - lava_callback - ERROR - Error processing callback for node 69c6b61d02375b4506df8be6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:54:14,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:14,198 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bff2866272ea51e2ad/log.txt.gz 2026-03-29 16:54:14,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:14,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:14,540 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bff2866272ea51e2ad/lava-logs.yaml 2026-03-29 16:54:14,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:14,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:14,730 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958bff2866272ea51e2ad/lava_callback.json.gz 2026-03-29 16:54:14,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:14,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958bff2866272ea51e2ad HTTP/1.1" 200 None 2026-03-29 16:54:14,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:14,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:14,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:14,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958bff2866272ea51e2ad HTTP/1.1" 200 None 2026-03-29 16:54:14,820 - lava_callback - INFO - Completed processing callback for node 69c958bff2866272ea51e2ad INFO: 172.20.0.1:45252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45264 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:54:16,643 - lava_callback - INFO - Received callback for node 69c6b62a02375b4506df8c23 INFO: 172.20.0.1:45270 - "POST /node/69c6b62a02375b4506df8c23 HTTP/1.0" 202 Accepted 2026-03-29 16:54:16,648 - lava_callback - INFO - Starting async job submit for node 69c6b62a02375b4506df8c23 2026-03-29 16:54:16,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:16,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b62a02375b4506df8c23 HTTP/1.1" 200 None 2026-03-29 16:54:16,664 - lava_callback - DEBUG - Processing node 69c6b62a02375b4506df8c23: baseline-arm64-baylibre 2026-03-29 16:54:16,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b62a02375b4506df8c23 2026-03-29 16:54:16,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:16,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b62a02375b4506df8c23 HTTP/1.1" 200 None 2026-03-29 16:54:16,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:16,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 16:54:16,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:16,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b62a02375b4506df8c23 HTTP/1.1" 200 None 2026-03-29 16:54:16,727 - lava_callback - ERROR - Error processing callback for node 69c6b62a02375b4506df8c23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:54:18,656 - lava_callback - INFO - Received callback for node 69c958dcf2866272ea51e308 INFO: 172.20.0.1:45282 - "POST /node/69c958dcf2866272ea51e308 HTTP/1.0" 202 Accepted 2026-03-29 16:54:18,663 - lava_callback - INFO - Starting async job submit for node 69c958dcf2866272ea51e308 2026-03-29 16:54:18,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:18,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958dcf2866272ea51e308 HTTP/1.1" 200 None 2026-03-29 16:54:18,709 - lava_callback - DEBUG - Processing node 69c958dcf2866272ea51e308: baseline-nfs-arm64 2026-03-29 16:54:19,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:19,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:19,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958dcf2866272ea51e308/log.txt.gz 2026-03-29 16:54:19,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:19,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:19,998 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958dcf2866272ea51e308/lava-logs.yaml 2026-03-29 16:54:20,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:20,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:20,172 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958dcf2866272ea51e308/lava_callback.json.gz 2026-03-29 16:54:20,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:20,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958dcf2866272ea51e308 HTTP/1.1" 200 None 2026-03-29 16:54:20,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:20,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:20,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:20,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958dcf2866272ea51e308 HTTP/1.1" 200 None 2026-03-29 16:54:20,264 - lava_callback - INFO - Completed processing callback for node 69c958dcf2866272ea51e308 2026-03-29 16:54:21,958 - lava_callback - INFO - Received callback for node 69c958daf2866272ea51e305 INFO: 172.20.0.1:39406 - "POST /node/69c958daf2866272ea51e305 HTTP/1.0" 202 Accepted 2026-03-29 16:54:21,965 - lava_callback - INFO - Starting async job submit for node 69c958daf2866272ea51e305 2026-03-29 16:54:21,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:22,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958daf2866272ea51e305 HTTP/1.1" 200 None 2026-03-29 16:54:22,015 - lava_callback - DEBUG - Processing node 69c958daf2866272ea51e305: baseline-nfs-arm64 2026-03-29 16:54:22,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:22,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:22,826 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958daf2866272ea51e305/log.txt.gz 2026-03-29 16:54:23,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:23,368 - lava_callback - INFO - Received callback for node 69c958d7f2866272ea51e2ff INFO: 172.20.0.1:39418 - "POST /node/69c958d7f2866272ea51e2ff HTTP/1.0" 202 Accepted 2026-03-29 16:54:23,376 - lava_callback - INFO - Starting async job submit for node 69c958d7f2866272ea51e2ff 2026-03-29 16:54:23,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:23,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:23,403 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958daf2866272ea51e305/lava-logs.yaml 2026-03-29 16:54:23,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:23,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d7f2866272ea51e2ff HTTP/1.1" 200 None 2026-03-29 16:54:23,419 - lava_callback - DEBUG - Processing node 69c958d7f2866272ea51e2ff: baseline-nfs-arm64 2026-03-29 16:54:23,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:23,695 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958daf2866272ea51e305/lava_callback.json.gz 2026-03-29 16:54:23,766 - lava_callback - INFO - Received callback for node 69c958e1f2866272ea51e318 2026-03-29 16:54:23,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:39430 - "POST /node/69c958e1f2866272ea51e318 HTTP/1.0" 202 Accepted 2026-03-29 16:54:23,806 - lava_callback - INFO - Starting async job submit for node 69c958e1f2866272ea51e318 2026-03-29 16:54:23,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:23,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:23,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958daf2866272ea51e305 HTTP/1.1" 200 None 2026-03-29 16:54:23,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:23,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e1f2866272ea51e318 HTTP/1.1" 200 None 2026-03-29 16:54:23,887 - lava_callback - DEBUG - Processing node 69c958e1f2866272ea51e318: baseline-nfs-arm64 2026-03-29 16:54:23,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:23,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:24,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958daf2866272ea51e305 HTTP/1.1" 200 None 2026-03-29 16:54:24,081 - lava_callback - INFO - Completed processing callback for node 69c958daf2866272ea51e305 2026-03-29 16:54:24,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:24,203 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d7f2866272ea51e2ff/log.txt.gz 2026-03-29 16:54:24,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:24,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:24,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:24,832 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e1f2866272ea51e318/log.txt.gz 2026-03-29 16:54:24,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:24,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d7f2866272ea51e2ff/lava-logs.yaml 2026-03-29 16:54:25,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:25,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:25,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:25,215 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d7f2866272ea51e2ff/lava_callback.json.gz 2026-03-29 16:54:25,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:25,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:25,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d7f2866272ea51e2ff HTTP/1.1" 200 None 2026-03-29 16:54:25,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:25,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e1f2866272ea51e318/lava-logs.yaml 2026-03-29 16:54:25,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:25,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:25,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:25,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958d7f2866272ea51e2ff HTTP/1.1" 200 None 2026-03-29 16:54:25,314 - lava_callback - INFO - Completed processing callback for node 69c958d7f2866272ea51e2ff 2026-03-29 16:54:25,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:25,485 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e1f2866272ea51e318/lava_callback.json.gz 2026-03-29 16:54:25,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:25,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e1f2866272ea51e318 HTTP/1.1" 200 None 2026-03-29 16:54:25,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:25,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:25,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:25,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958e1f2866272ea51e318 HTTP/1.1" 200 None 2026-03-29 16:54:25,580 - lava_callback - INFO - Completed processing callback for node 69c958e1f2866272ea51e318 INFO: 172.20.0.1:39434 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:54:27,575 - lava_callback - INFO - Received callback for node 69c958def2866272ea51e312 INFO: 172.20.0.1:39444 - "POST /node/69c958def2866272ea51e312 HTTP/1.0" 202 Accepted 2026-03-29 16:54:27,582 - lava_callback - INFO - Starting async job submit for node 69c958def2866272ea51e312 2026-03-29 16:54:27,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:27,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958def2866272ea51e312 HTTP/1.1" 200 None 2026-03-29 16:54:27,622 - lava_callback - DEBUG - Processing node 69c958def2866272ea51e312: baseline-nfs-arm64 2026-03-29 16:54:27,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:28,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:28,089 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958def2866272ea51e312/log.txt.gz 2026-03-29 16:54:28,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:28,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:28,503 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958def2866272ea51e312/lava-logs.yaml 2026-03-29 16:54:28,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:28,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:28,645 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958def2866272ea51e312/lava_callback.json.gz 2026-03-29 16:54:28,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:28,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958def2866272ea51e312 HTTP/1.1" 200 None 2026-03-29 16:54:28,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:28,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:28,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:28,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958def2866272ea51e312 HTTP/1.1" 200 None 2026-03-29 16:54:28,744 - lava_callback - INFO - Completed processing callback for node 69c958def2866272ea51e312 2026-03-29 16:54:29,070 - lava_callback - INFO - Received callback for node 69c9590cf2866272ea51e3b2 INFO: 172.20.0.1:39456 - "POST /node/69c9590cf2866272ea51e3b2 HTTP/1.0" 202 Accepted 2026-03-29 16:54:29,077 - lava_callback - INFO - Starting async job submit for node 69c9590cf2866272ea51e3b2 2026-03-29 16:54:29,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:29,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9590cf2866272ea51e3b2 HTTP/1.1" 200 None 2026-03-29 16:54:29,114 - lava_callback - DEBUG - Processing node 69c9590cf2866272ea51e3b2: baseline-arm64-pengutronix 2026-03-29 16:54:29,216 - lava_callback - INFO - Received callback for node 69c958d5f2866272ea51e2f9 INFO: 172.20.0.1:39470 - "POST /node/69c958d5f2866272ea51e2f9 HTTP/1.0" 202 Accepted 2026-03-29 16:54:29,243 - lava_callback - INFO - Starting async job submit for node 69c958d5f2866272ea51e2f9 2026-03-29 16:54:29,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:29,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:29,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d5f2866272ea51e2f9 HTTP/1.1" 200 None 2026-03-29 16:54:29,352 - lava_callback - DEBUG - Processing node 69c958d5f2866272ea51e2f9: baseline-nfs-arm64 2026-03-29 16:54:29,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:29,574 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590cf2866272ea51e3b2/log.txt.gz 2026-03-29 16:54:29,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:29,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:29,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:29,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 16:54:29,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:29,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d5f2866272ea51e2f9/log.txt.gz 2026-03-29 16:54:29,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:30,037 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590cf2866272ea51e3b2/lava-logs.yaml 2026-03-29 16:54:30,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:30,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:30,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:30,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590cf2866272ea51e3b2/lava_callback.json.gz 2026-03-29 16:54:30,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:30,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9590cf2866272ea51e3b2 HTTP/1.1" 200 None 2026-03-29 16:54:30,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:30,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:30,358 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d5f2866272ea51e2f9/lava-logs.yaml 2026-03-29 16:54:30,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:30,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:30,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:30,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9590cf2866272ea51e3b2 HTTP/1.1" 200 None 2026-03-29 16:54:30,418 - lava_callback - INFO - Completed processing callback for node 69c9590cf2866272ea51e3b2 2026-03-29 16:54:30,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:30,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d5f2866272ea51e2f9/lava_callback.json.gz 2026-03-29 16:54:30,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:30,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d5f2866272ea51e2f9 HTTP/1.1" 200 None 2026-03-29 16:54:30,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:30,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:30,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:30,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958d5f2866272ea51e2f9 HTTP/1.1" 200 None 2026-03-29 16:54:30,715 - lava_callback - INFO - Completed processing callback for node 69c958d5f2866272ea51e2f9 2026-03-29 16:54:32,904 - lava_callback - INFO - Received callback for node 69c95911f2866272ea51e3c1 INFO: 172.20.0.1:50858 - "POST /node/69c95911f2866272ea51e3c1 HTTP/1.0" 202 Accepted 2026-03-29 16:54:32,911 - lava_callback - INFO - Starting async job submit for node 69c95911f2866272ea51e3c1 2026-03-29 16:54:32,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:32,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95911f2866272ea51e3c1 HTTP/1.1" 200 None 2026-03-29 16:54:32,960 - lava_callback - DEBUG - Processing node 69c95911f2866272ea51e3c1: baseline-arm64-foundriesio 2026-03-29 16:54:33,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:33,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:33,346 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c95911f2866272ea51e3c1/log.txt.gz 2026-03-29 16:54:33,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:33,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:33,641 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c95911f2866272ea51e3c1/lava-logs.yaml 2026-03-29 16:54:33,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:33,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:33,777 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c95911f2866272ea51e3c1/lava_callback.json.gz 2026-03-29 16:54:33,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:33,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95911f2866272ea51e3c1 HTTP/1.1" 200 None 2026-03-29 16:54:33,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:33,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:33,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:33,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95911f2866272ea51e3c1 HTTP/1.1" 200 None 2026-03-29 16:54:33,858 - lava_callback - INFO - Completed processing callback for node 69c95911f2866272ea51e3c1 2026-03-29 16:54:35,592 - lava_callback - INFO - Received callback for node 69c958e3f2866272ea51e31b INFO: 172.20.0.1:50874 - "POST /node/69c958e3f2866272ea51e31b HTTP/1.0" 202 Accepted 2026-03-29 16:54:35,602 - lava_callback - INFO - Starting async job submit for node 69c958e3f2866272ea51e31b 2026-03-29 16:54:35,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:35,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e3f2866272ea51e31b HTTP/1.1" 200 None 2026-03-29 16:54:35,650 - lava_callback - DEBUG - Processing node 69c958e3f2866272ea51e31b: baseline-nfs-arm64 2026-03-29 16:54:35,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:36,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:36,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e3f2866272ea51e31b/log.txt.gz 2026-03-29 16:54:36,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:54:36,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:36,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e3f2866272ea51e31b/lava-logs.yaml 2026-03-29 16:54:36,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:36,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:36,711 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e3f2866272ea51e31b/lava_callback.json.gz 2026-03-29 16:54:36,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:36,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e3f2866272ea51e31b HTTP/1.1" 200 None 2026-03-29 16:54:36,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:36,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:36,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:36,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958e3f2866272ea51e31b HTTP/1.1" 200 None 2026-03-29 16:54:36,803 - lava_callback - INFO - Completed processing callback for node 69c958e3f2866272ea51e31b 2026-03-29 16:54:37,427 - lava_callback - INFO - Received callback for node 69c958e4f2866272ea51e31e INFO: 172.20.0.1:50882 - "POST /node/69c958e4f2866272ea51e31e HTTP/1.0" 202 Accepted 2026-03-29 16:54:37,434 - lava_callback - INFO - Starting async job submit for node 69c958e4f2866272ea51e31e 2026-03-29 16:54:37,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:37,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e4f2866272ea51e31e HTTP/1.1" 200 None 2026-03-29 16:54:37,491 - lava_callback - DEBUG - Processing node 69c958e4f2866272ea51e31e: baseline-nfs-arm64 2026-03-29 16:54:37,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:38,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:38,089 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e4f2866272ea51e31e/log.txt.gz 2026-03-29 16:54:38,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:38,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:38,547 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e4f2866272ea51e31e/lava-logs.yaml 2026-03-29 16:54:38,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:38,561 - lava_callback - INFO - Received callback for node 69c958f2f2866272ea51e35c INFO: 172.20.0.1:50892 - "POST /node/69c958f2f2866272ea51e35c HTTP/1.0" 202 Accepted 2026-03-29 16:54:38,571 - lava_callback - INFO - Starting async job submit for node 69c958f2f2866272ea51e35c 2026-03-29 16:54:38,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:38,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f2f2866272ea51e35c HTTP/1.1" 200 None 2026-03-29 16:54:38,628 - lava_callback - DEBUG - Processing node 69c958f2f2866272ea51e35c: baseline-arm64-broonie 2026-03-29 16:54:38,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:38,853 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e4f2866272ea51e31e/lava_callback.json.gz 2026-03-29 16:54:38,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:38,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:38,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e4f2866272ea51e31e HTTP/1.1" 200 None 2026-03-29 16:54:38,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:38,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:38,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:38,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958e4f2866272ea51e31e HTTP/1.1" 200 None 2026-03-29 16:54:38,980 - lava_callback - INFO - Completed processing callback for node 69c958e4f2866272ea51e31e 2026-03-29 16:54:39,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:39,065 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f2f2866272ea51e35c/log.txt.gz 2026-03-29 16:54:39,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:39,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:39,474 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f2f2866272ea51e35c/lava-logs.yaml 2026-03-29 16:54:39,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:39,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:39,633 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f2f2866272ea51e35c/lava_callback.json.gz 2026-03-29 16:54:39,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:39,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f2f2866272ea51e35c HTTP/1.1" 200 None 2026-03-29 16:54:39,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:39,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:39,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:39,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958f2f2866272ea51e35c HTTP/1.1" 200 None 2026-03-29 16:54:39,720 - lava_callback - INFO - Completed processing callback for node 69c958f2f2866272ea51e35c 2026-03-29 16:54:40,117 - lava_callback - INFO - Received callback for node 69c958faf2866272ea51e36f INFO: 172.20.0.1:46984 - "POST /node/69c958faf2866272ea51e36f HTTP/1.0" 202 Accepted 2026-03-29 16:54:40,124 - lava_callback - INFO - Starting async job submit for node 69c958faf2866272ea51e36f 2026-03-29 16:54:40,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:40,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958faf2866272ea51e36f HTTP/1.1" 200 None 2026-03-29 16:54:40,164 - lava_callback - DEBUG - Processing node 69c958faf2866272ea51e36f: baseline-arm64-broonie 2026-03-29 16:54:40,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:40,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:40,544 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958faf2866272ea51e36f/log.txt.gz 2026-03-29 16:54:40,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:40,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:40,947 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958faf2866272ea51e36f/lava-logs.yaml 2026-03-29 16:54:40,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:41,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:41,104 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958faf2866272ea51e36f/lava_callback.json.gz 2026-03-29 16:54:41,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:41,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958faf2866272ea51e36f HTTP/1.1" 200 None 2026-03-29 16:54:41,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:41,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:41,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:41,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958faf2866272ea51e36f HTTP/1.1" 200 None 2026-03-29 16:54:41,189 - lava_callback - INFO - Completed processing callback for node 69c958faf2866272ea51e36f INFO: 172.20.0.1:46994 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:54:54,140 - lava_callback - INFO - Received callback for node 69c9590df2866272ea51e3b5 INFO: 172.20.0.1:35590 - "POST /node/69c9590df2866272ea51e3b5 HTTP/1.0" 202 Accepted 2026-03-29 16:54:54,147 - lava_callback - INFO - Starting async job submit for node 69c9590df2866272ea51e3b5 2026-03-29 16:54:54,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:54,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9590df2866272ea51e3b5 HTTP/1.1" 200 None 2026-03-29 16:54:54,195 - lava_callback - DEBUG - Processing node 69c9590df2866272ea51e3b5: baseline-arm64-pengutronix 2026-03-29 16:54:54,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:54,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:54,527 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590df2866272ea51e3b5/log.txt.gz 2026-03-29 16:54:54,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:54,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:54,818 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590df2866272ea51e3b5/lava-logs.yaml 2026-03-29 16:54:54,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:54:55,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:54:55,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590df2866272ea51e3b5/lava_callback.json.gz 2026-03-29 16:54:55,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:55,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9590df2866272ea51e3b5 HTTP/1.1" 200 None 2026-03-29 16:54:55,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:55,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:54:55,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:55,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9590df2866272ea51e3b5 HTTP/1.1" 200 None 2026-03-29 16:54:55,100 - lava_callback - INFO - Completed processing callback for node 69c9590df2866272ea51e3b5 INFO: 172.20.0.1:35606 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:54:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:54:59,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 16:55:03,596 - lava_callback - INFO - Received callback for node 69c9590ff2866272ea51e3bc INFO: 172.20.0.1:40168 - "POST /node/69c9590ff2866272ea51e3bc HTTP/1.0" 202 Accepted 2026-03-29 16:55:03,602 - lava_callback - INFO - Starting async job submit for node 69c9590ff2866272ea51e3bc 2026-03-29 16:55:03,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:03,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9590ff2866272ea51e3bc HTTP/1.1" 200 None 2026-03-29 16:55:03,638 - lava_callback - DEBUG - Processing node 69c9590ff2866272ea51e3bc: baseline-arm64-pengutronix 2026-03-29 16:55:03,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:03,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:03,966 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590ff2866272ea51e3bc/log.txt.gz 2026-03-29 16:55:04,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:04,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:04,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590ff2866272ea51e3bc/lava-logs.yaml 2026-03-29 16:55:04,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:04,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:04,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c9590ff2866272ea51e3bc/lava_callback.json.gz 2026-03-29 16:55:04,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:04,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9590ff2866272ea51e3bc HTTP/1.1" 200 None 2026-03-29 16:55:04,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:04,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:55:04,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:04,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9590ff2866272ea51e3bc HTTP/1.1" 200 None 2026-03-29 16:55:04,557 - lava_callback - INFO - Completed processing callback for node 69c9590ff2866272ea51e3bc INFO: 172.20.0.1:40172 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:55:07,888 - lava_callback - INFO - Received callback for node 69c956a3f2866272ea51e10a INFO: 172.20.0.1:40180 - "POST /node/69c956a3f2866272ea51e10a HTTP/1.0" 202 Accepted 2026-03-29 16:55:07,894 - lava_callback - INFO - Starting async job submit for node 69c956a3f2866272ea51e10a 2026-03-29 16:55:07,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:07,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a3f2866272ea51e10a HTTP/1.1" 200 None 2026-03-29 16:55:07,936 - lava_callback - DEBUG - Processing node 69c956a3f2866272ea51e10a: ltp-sched 2026-03-29 16:55:08,084 - lava_callback - INFO - Received callback for node 69c95907f2866272ea51e3a3 INFO: 172.20.0.1:40192 - "POST /node/69c95907f2866272ea51e3a3 HTTP/1.0" 202 Accepted 2026-03-29 16:55:08,101 - lava_callback - INFO - Starting async job submit for node 69c95907f2866272ea51e3a3 2026-03-29 16:55:08,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:08,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95907f2866272ea51e3a3 HTTP/1.1" 200 None 2026-03-29 16:55:08,229 - lava_callback - DEBUG - Processing node 69c95907f2866272ea51e3a3: baseline-arm64-clabbe 2026-03-29 16:55:08,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:08,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:08,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:08,762 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c956a3f2866272ea51e10a/log.txt.gz 2026-03-29 16:55:08,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:08,797 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c95907f2866272ea51e3a3/log.txt.gz 2026-03-29 16:55:09,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:09,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:09,314 - lava_callback - INFO - Received callback for node 69c958f5f2866272ea51e366 INFO: 172.20.0.1:40196 - "POST /node/69c958f5f2866272ea51e366 HTTP/1.0" 202 Accepted 2026-03-29 16:55:09,322 - lava_callback - INFO - Starting async job submit for node 69c958f5f2866272ea51e366 2026-03-29 16:55:09,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:09,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:09,349 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c95907f2866272ea51e3a3/lava-logs.yaml 2026-03-29 16:55:09,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:09,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f5f2866272ea51e366 HTTP/1.1" 200 None 2026-03-29 16:55:09,366 - lava_callback - DEBUG - Processing node 69c958f5f2866272ea51e366: baseline-arm64-broonie 2026-03-29 16:55:09,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:09,468 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c956a3f2866272ea51e10a/lava-logs.yaml 2026-03-29 16:55:09,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:09,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:09,625 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c95907f2866272ea51e3a3/lava_callback.json.gz 2026-03-29 16:55:09,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:09,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:09,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95907f2866272ea51e3a3 HTTP/1.1" 200 None 2026-03-29 16:55:09,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:09,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:55:09,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:09,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95907f2866272ea51e3a3 HTTP/1.1" 200 None 2026-03-29 16:55:09,776 - lava_callback - INFO - Completed processing callback for node 69c95907f2866272ea51e3a3 2026-03-29 16:55:09,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:09,889 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f5f2866272ea51e366/log.txt.gz 2026-03-29 16:55:09,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:09,964 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c956a3f2866272ea51e10a/lava_callback.json.gz 2026-03-29 16:55:10,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:10,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a3f2866272ea51e10a HTTP/1.1" 200 None 2026-03-29 16:55:10,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:10,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:10,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:55:10,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:10,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956a3f2866272ea51e10a HTTP/1.1" 200 None 2026-03-29 16:55:10,248 - lava_callback - INFO - Completed processing callback for node 69c956a3f2866272ea51e10a 2026-03-29 16:55:10,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:10,448 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f5f2866272ea51e366/lava-logs.yaml 2026-03-29 16:55:10,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:10,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:10,595 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f5f2866272ea51e366/lava_callback.json.gz 2026-03-29 16:55:10,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:10,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f5f2866272ea51e366 HTTP/1.1" 200 None 2026-03-29 16:55:10,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:10,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:55:10,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:10,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958f5f2866272ea51e366 HTTP/1.1" 200 None 2026-03-29 16:55:10,689 - lava_callback - INFO - Completed processing callback for node 69c958f5f2866272ea51e366 INFO: 172.20.0.1:34688 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:55:11,054 - lava_callback - INFO - Received callback for node 69c958fef2866272ea51e375 INFO: 172.20.0.1:34690 - "POST /node/69c958fef2866272ea51e375 HTTP/1.0" 202 Accepted 2026-03-29 16:55:11,061 - lava_callback - INFO - Starting async job submit for node 69c958fef2866272ea51e375 2026-03-29 16:55:11,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:11,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958fef2866272ea51e375 HTTP/1.1" 200 None 2026-03-29 16:55:11,101 - lava_callback - DEBUG - Processing node 69c958fef2866272ea51e375: baseline-arm64-broonie 2026-03-29 16:55:11,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:11,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:11,481 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958fef2866272ea51e375/log.txt.gz 2026-03-29 16:55:11,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:11,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:11,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958fef2866272ea51e375/lava-logs.yaml 2026-03-29 16:55:11,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:11,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:11,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958fef2866272ea51e375/lava_callback.json.gz 2026-03-29 16:55:12,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:12,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958fef2866272ea51e375 HTTP/1.1" 200 None 2026-03-29 16:55:12,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:12,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:55:12,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:12,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958fef2866272ea51e375 HTTP/1.1" 200 None 2026-03-29 16:55:12,079 - lava_callback - INFO - Completed processing callback for node 69c958fef2866272ea51e375 INFO: 172.20.0.1:34702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34716 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:55:18,833 - lava_callback - INFO - Received callback for node 69c956a7f2866272ea51e113 INFO: 172.20.0.1:34720 - "POST /node/69c956a7f2866272ea51e113 HTTP/1.0" 202 Accepted 2026-03-29 16:55:18,842 - lava_callback - INFO - Starting async job submit for node 69c956a7f2866272ea51e113 2026-03-29 16:55:18,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:18,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a7f2866272ea51e113 HTTP/1.1" 200 None 2026-03-29 16:55:18,911 - lava_callback - DEBUG - Processing node 69c956a7f2866272ea51e113: ltp-syscalls-ipc 2026-03-29 16:55:19,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:20,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:20,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c956a7f2866272ea51e113/log.txt.gz 2026-03-29 16:55:20,330 - lava_callback - INFO - Received callback for node 69c95900f2866272ea51e378 INFO: 172.20.0.1:53466 - "POST /node/69c95900f2866272ea51e378 HTTP/1.0" 202 Accepted 2026-03-29 16:55:20,344 - lava_callback - INFO - Starting async job submit for node 69c95900f2866272ea51e378 2026-03-29 16:55:20,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:20,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:20,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95900f2866272ea51e378 HTTP/1.1" 200 None 2026-03-29 16:55:20,408 - lava_callback - DEBUG - Processing node 69c95900f2866272ea51e378: baseline-arm64-broonie 2026-03-29 16:55:20,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:21,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:21,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c956a7f2866272ea51e113/lava-logs.yaml 2026-03-29 16:55:21,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:21,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:21,422 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c95900f2866272ea51e378/log.txt.gz 2026-03-29 16:55:21,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:21,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:21,852 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c956a7f2866272ea51e113/lava_callback.json.gz 2026-03-29 16:55:21,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:21,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a7f2866272ea51e113 HTTP/1.1" 200 None 2026-03-29 16:55:21,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:21,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:55:21,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:21,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956a7f2866272ea51e113 HTTP/1.1" 200 None 2026-03-29 16:55:22,009 - lava_callback - INFO - Completed processing callback for node 69c956a7f2866272ea51e113 2026-03-29 16:55:22,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:22,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c95900f2866272ea51e378/lava-logs.yaml 2026-03-29 16:55:22,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:23,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:23,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c95900f2866272ea51e378/lava_callback.json.gz 2026-03-29 16:55:23,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:23,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95900f2866272ea51e378 HTTP/1.1" 200 None 2026-03-29 16:55:23,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:23,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:55:23,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:23,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95900f2866272ea51e378 HTTP/1.1" 200 None 2026-03-29 16:55:23,166 - lava_callback - INFO - Completed processing callback for node 69c95900f2866272ea51e378 INFO: 172.20.0.1:53474 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:55:27,869 - lava_callback - INFO - Received callback for node 69c956a4f2866272ea51e10d INFO: 172.20.0.1:53478 - "POST /node/69c956a4f2866272ea51e10d HTTP/1.0" 202 Accepted 2026-03-29 16:55:27,878 - lava_callback - INFO - Starting async job submit for node 69c956a4f2866272ea51e10d 2026-03-29 16:55:27,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a4f2866272ea51e10d HTTP/1.1" 200 None 2026-03-29 16:55:27,931 - lava_callback - DEBUG - Processing node 69c956a4f2866272ea51e10d: ltp-smoke 2026-03-29 16:55:28,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:28,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:28,949 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c956a4f2866272ea51e10d/log.txt.gz 2026-03-29 16:55:29,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:29,448 - lava_callback - INFO - Received callback for node 69c958f7f2866272ea51e369 INFO: 172.20.0.1:53492 - "POST /node/69c958f7f2866272ea51e369 HTTP/1.0" 202 Accepted 2026-03-29 16:55:29,458 - lava_callback - INFO - Starting async job submit for node 69c958f7f2866272ea51e369 2026-03-29 16:55:29,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:29,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f7f2866272ea51e369 HTTP/1.1" 200 None 2026-03-29 16:55:29,548 - lava_callback - DEBUG - Processing node 69c958f7f2866272ea51e369: baseline-arm64-broonie 2026-03-29 16:55:29,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:30,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 16:55:30,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:30,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:30,506 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c956a4f2866272ea51e10d/lava-logs.yaml 2026-03-29 16:55:30,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:30,869 - lava_callback - INFO - Received callback for node 69c958fcf2866272ea51e372 INFO: 172.20.0.1:45528 - "POST /node/69c958fcf2866272ea51e372 HTTP/1.0" 202 Accepted 2026-03-29 16:55:30,876 - lava_callback - INFO - Starting async job submit for node 69c958fcf2866272ea51e372 2026-03-29 16:55:30,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:30,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958fcf2866272ea51e372 HTTP/1.1" 200 None 2026-03-29 16:55:30,962 - lava_callback - DEBUG - Processing node 69c958fcf2866272ea51e372: baseline-arm64-broonie 2026-03-29 16:55:31,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:31,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:31,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f7f2866272ea51e369/log.txt.gz 2026-03-29 16:55:31,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:31,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:31,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c956a4f2866272ea51e10d/lava_callback.json.gz 2026-03-29 16:55:31,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:31,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a4f2866272ea51e10d HTTP/1.1" 200 None 2026-03-29 16:55:31,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:31,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:55:31,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:31,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956a4f2866272ea51e10d HTTP/1.1" 200 None 2026-03-29 16:55:31,942 - lava_callback - INFO - Completed processing callback for node 69c956a4f2866272ea51e10d 2026-03-29 16:55:32,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:32,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958fcf2866272ea51e372/log.txt.gz 2026-03-29 16:55:32,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:33,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:33,046 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f7f2866272ea51e369/lava-logs.yaml 2026-03-29 16:55:33,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:34,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:34,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958fcf2866272ea51e372/lava-logs.yaml 2026-03-29 16:55:34,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:55:34,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:34,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f7f2866272ea51e369/lava_callback.json.gz 2026-03-29 16:55:34,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:34,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f7f2866272ea51e369 HTTP/1.1" 200 None 2026-03-29 16:55:34,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:34,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:55:34,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:34,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958f7f2866272ea51e369 HTTP/1.1" 200 None 2026-03-29 16:55:34,480 - lava_callback - INFO - Completed processing callback for node 69c958f7f2866272ea51e369 2026-03-29 16:55:35,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:55:35,392 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958fcf2866272ea51e372/lava_callback.json.gz 2026-03-29 16:55:35,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:35,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958fcf2866272ea51e372 HTTP/1.1" 200 None 2026-03-29 16:55:35,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:35,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:55:35,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:35,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958fcf2866272ea51e372 HTTP/1.1" 200 None 2026-03-29 16:55:35,502 - lava_callback - INFO - Completed processing callback for node 69c958fcf2866272ea51e372 INFO: 172.20.0.1:45538 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:55:36,718 - lava_callback - INFO - Received callback for node 69c6b67a02375b4506df8cb8 INFO: 172.20.0.1:45546 - "POST /node/69c6b67a02375b4506df8cb8 HTTP/1.0" 202 Accepted 2026-03-29 16:55:36,728 - lava_callback - INFO - Starting async job submit for node 69c6b67a02375b4506df8cb8 2026-03-29 16:55:36,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:36,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b67a02375b4506df8cb8 HTTP/1.1" 200 None 2026-03-29 16:55:36,748 - lava_callback - DEBUG - Processing node 69c6b67a02375b4506df8cb8: baseline-arm64-baylibre 2026-03-29 16:55:36,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b67a02375b4506df8cb8 2026-03-29 16:55:36,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:36,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b67a02375b4506df8cb8 HTTP/1.1" 200 None 2026-03-29 16:55:36,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:36,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 16:55:36,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:36,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b67a02375b4506df8cb8 HTTP/1.1" 200 None 2026-03-29 16:55:36,828 - lava_callback - ERROR - Error processing callback for node 69c6b67a02375b4506df8cb8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:55:58,866 - lava_callback - INFO - Received callback for node 69c958f3f2866272ea51e35f INFO: 172.20.0.1:53726 - "POST /node/69c958f3f2866272ea51e35f HTTP/1.0" 202 Accepted 2026-03-29 16:55:58,875 - lava_callback - INFO - Starting async job submit for node 69c958f3f2866272ea51e35f 2026-03-29 16:55:58,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:55:58,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f3f2866272ea51e35f HTTP/1.1" 200 None 2026-03-29 16:55:58,946 - lava_callback - DEBUG - Processing node 69c958f3f2866272ea51e35f: baseline-arm64-broonie 2026-03-29 16:55:59,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:00,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:00,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 16:56:00,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:00,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f3f2866272ea51e35f/log.txt.gz 2026-03-29 16:56:00,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:02,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:02,167 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f3f2866272ea51e35f/lava-logs.yaml 2026-03-29 16:56:02,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:02,760 - lava_callback - INFO - Received callback for node 69c958c4f2866272ea51e2b6 INFO: 172.20.0.1:45334 - "POST /node/69c958c4f2866272ea51e2b6 HTTP/1.0" 202 Accepted 2026-03-29 16:56:02,770 - lava_callback - INFO - Starting async job submit for node 69c958c4f2866272ea51e2b6 2026-03-29 16:56:02,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:02,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c4f2866272ea51e2b6 HTTP/1.1" 200 None 2026-03-29 16:56:02,822 - lava_callback - DEBUG - Processing node 69c958c4f2866272ea51e2b6: baseline-arm64 2026-03-29 16:56:03,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:03,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f3f2866272ea51e35f/lava_callback.json.gz 2026-03-29 16:56:03,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:03,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:03,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f3f2866272ea51e35f HTTP/1.1" 200 None 2026-03-29 16:56:03,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:03,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:56:03,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:03,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958f3f2866272ea51e35f HTTP/1.1" 200 None 2026-03-29 16:56:03,422 - lava_callback - INFO - Completed processing callback for node 69c958f3f2866272ea51e35f 2026-03-29 16:56:04,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:04,610 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c4f2866272ea51e2b6/log.txt.gz 2026-03-29 16:56:04,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:06,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:06,231 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c4f2866272ea51e2b6/lava-logs.yaml 2026-03-29 16:56:06,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45348 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:56:07,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:07,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958c4f2866272ea51e2b6/lava_callback.json.gz 2026-03-29 16:56:07,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:07,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c4f2866272ea51e2b6 HTTP/1.1" 200 None 2026-03-29 16:56:07,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:07,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:56:07,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:07,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c4f2866272ea51e2b6 HTTP/1.1" 200 None 2026-03-29 16:56:07,469 - lava_callback - INFO - Completed processing callback for node 69c958c4f2866272ea51e2b6 INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59292 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:56:30,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:30,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:56:38,688 - lava_callback - INFO - Received callback for node 69c958f8f2866272ea51e36c INFO: 172.20.0.1:50890 - "POST /node/69c958f8f2866272ea51e36c HTTP/1.0" 202 Accepted 2026-03-29 16:56:38,694 - lava_callback - INFO - Starting async job submit for node 69c958f8f2866272ea51e36c 2026-03-29 16:56:38,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:38,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f8f2866272ea51e36c HTTP/1.1" 200 None 2026-03-29 16:56:38,736 - lava_callback - DEBUG - Processing node 69c958f8f2866272ea51e36c: baseline-arm64-broonie 2026-03-29 16:56:39,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:40,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:40,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f8f2866272ea51e36c/log.txt.gz 2026-03-29 16:56:40,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:42,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:42,680 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f8f2866272ea51e36c/lava-logs.yaml 2026-03-29 16:56:42,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:56:44,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:56:44,253 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c958f8f2866272ea51e36c/lava_callback.json.gz 2026-03-29 16:56:44,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:44,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958f8f2866272ea51e36c HTTP/1.1" 200 None 2026-03-29 16:56:44,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:44,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:56:44,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:44,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958f8f2866272ea51e36c HTTP/1.1" 200 None 2026-03-29 16:56:44,355 - lava_callback - INFO - Completed processing callback for node 69c958f8f2866272ea51e36c INFO: 172.20.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:56:56,884 - lava_callback - INFO - Received callback for node 69c6b6cf02375b4506dfb3cc INFO: 172.20.0.1:59130 - "POST /node/69c6b6cf02375b4506dfb3cc HTTP/1.0" 202 Accepted 2026-03-29 16:56:56,894 - lava_callback - INFO - Starting async job submit for node 69c6b6cf02375b4506dfb3cc 2026-03-29 16:56:56,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:56,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b6cf02375b4506dfb3cc HTTP/1.1" 200 None 2026-03-29 16:56:56,917 - lava_callback - DEBUG - Processing node 69c6b6cf02375b4506dfb3cc: baseline-arm64-baylibre 2026-03-29 16:56:56,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b6cf02375b4506dfb3cc 2026-03-29 16:56:56,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:56,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b6cf02375b4506dfb3cc HTTP/1.1" 200 None 2026-03-29 16:56:56,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:56,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 16:56:56,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:56:56,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b6cf02375b4506dfb3cc HTTP/1.1" 200 None 2026-03-29 16:56:57,001 - lava_callback - ERROR - Error processing callback for node 69c6b6cf02375b4506dfb3cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:57:00,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:57:00,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47622 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:57:08,041 - lava_callback - INFO - Received callback for node 69c956aaf2866272ea51e11e INFO: 172.20.0.1:47636 - "POST /node/69c956aaf2866272ea51e11e HTTP/1.0" 202 Accepted 2026-03-29 16:57:08,052 - lava_callback - INFO - Starting async job submit for node 69c956aaf2866272ea51e11e 2026-03-29 16:57:08,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:57:08,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956aaf2866272ea51e11e HTTP/1.1" 200 None 2026-03-29 16:57:08,117 - lava_callback - DEBUG - Processing node 69c956aaf2866272ea51e11e: ltp-watchqueue 2026-03-29 16:57:08,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:57:10,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:57:10,306 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c956aaf2866272ea51e11e/log.txt.gz 2026-03-29 16:57:10,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:46928 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:57:12,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:57:12,335 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c956aaf2866272ea51e11e/lava-logs.yaml 2026-03-29 16:57:12,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:57:13,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:57:14,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c956aaf2866272ea51e11e/lava_callback.json.gz 2026-03-29 16:57:14,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:57:14,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956aaf2866272ea51e11e HTTP/1.1" 200 None 2026-03-29 16:57:14,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:57:14,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:57:14,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:57:14,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956aaf2866272ea51e11e HTTP/1.1" 200 None 2026-03-29 16:57:14,147 - lava_callback - INFO - Completed processing callback for node 69c956aaf2866272ea51e11e INFO: 172.20.0.1:46938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59820 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:57:30,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:57:30,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48648 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:58:05,060 - lava_callback - INFO - Received callback for node 69c6b71402375b4506dfb3f4 INFO: 172.20.0.1:48506 - "POST /node/69c6b71402375b4506dfb3f4 HTTP/1.0" 202 Accepted 2026-03-29 16:58:05,068 - lava_callback - INFO - Starting async job submit for node 69c6b71402375b4506dfb3f4 2026-03-29 16:58:05,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:05,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b71402375b4506dfb3f4 HTTP/1.1" 200 None 2026-03-29 16:58:05,093 - lava_callback - DEBUG - Processing node 69c6b71402375b4506dfb3f4: baseline-arm-clabbe 2026-03-29 16:58:05,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b71402375b4506dfb3f4 2026-03-29 16:58:05,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:05,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b71402375b4506dfb3f4 HTTP/1.1" 200 None 2026-03-29 16:58:05,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:05,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 16:58:05,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:05,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b71402375b4506dfb3f4 HTTP/1.1" 200 None 2026-03-29 16:58:05,182 - lava_callback - ERROR - Error processing callback for node 69c6b71402375b4506dfb3f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:58:31,255 - lava_callback - INFO - Received callback for node 69c95912f2866272ea51e3c6 INFO: 172.20.0.1:33332 - "POST /node/69c95912f2866272ea51e3c6 HTTP/1.0" 202 Accepted 2026-03-29 16:58:31,272 - lava_callback - INFO - Starting async job submit for node 69c95912f2866272ea51e3c6 2026-03-29 16:58:31,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:31,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95912f2866272ea51e3c6 HTTP/1.1" 200 None 2026-03-29 16:58:31,377 - lava_callback - DEBUG - Processing node 69c95912f2866272ea51e3c6: baseline-arm64-foundriesio 2026-03-29 16:58:33,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:58:34,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:58:34,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c95912f2866272ea51e3c6/log.txt.gz 2026-03-29 16:58:35,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33344 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:58:37,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:58:37,789 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c95912f2866272ea51e3c6/lava-logs.yaml 2026-03-29 16:58:37,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:58:39,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:58:39,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c95912f2866272ea51e3c6/lava_callback.json.gz DEBUG: 69c95912f2866272ea51e3c6 Transitting job node result: pass -> incomplete 2026-03-29 16:58:39,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:39,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95912f2866272ea51e3c6 HTTP/1.1" 200 None 2026-03-29 16:58:39,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:39,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 16:58:39,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:39,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95912f2866272ea51e3c6 HTTP/1.1" 200 None 2026-03-29 16:58:39,576 - lava_callback - INFO - Completed processing callback for node 69c95912f2866272ea51e3c6 2026-03-29 16:58:44,804 - lava_callback - INFO - Received callback for node 69c9565ef2866272ea51e02f INFO: 172.20.0.1:47852 - "POST /node/69c9565ef2866272ea51e02f HTTP/1.0" 202 Accepted 2026-03-29 16:58:44,811 - lava_callback - INFO - Starting async job submit for node 69c9565ef2866272ea51e02f 2026-03-29 16:58:44,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:44,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565ef2866272ea51e02f HTTP/1.1" 200 None 2026-03-29 16:58:44,928 - lava_callback - DEBUG - Processing node 69c9565ef2866272ea51e02f: baseline-arm 2026-03-29 16:58:45,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47860 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:58:47,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:58:47,339 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565ef2866272ea51e02f/log.txt.gz 2026-03-29 16:58:47,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:58:49,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:58:49,459 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565ef2866272ea51e02f/lava-logs.yaml 2026-03-29 16:58:49,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 16:58:51,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 16:58:51,102 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c9565ef2866272ea51e02f/lava_callback.json.gz DEBUG: Setup failed for 69c9565ef2866272ea51e02f. Transitting job node result: incomplete -> fail 2026-03-29 16:58:51,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:51,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9565ef2866272ea51e02f HTTP/1.1" 200 None 2026-03-29 16:58:51,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:51,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 16:58:51,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:51,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9565ef2866272ea51e02f HTTP/1.1" 200 None 2026-03-29 16:58:51,503 - lava_callback - INFO - Completed processing callback for node 69c9565ef2866272ea51e02f INFO: 172.20.0.1:56822 - "GET / HTTP/1.0" 200 OK 2026-03-29 16:58:56,972 - lava_callback - INFO - Received callback for node 69c6b74002375b4506dfb404 INFO: 172.20.0.1:56832 - "POST /node/69c6b74002375b4506dfb404 HTTP/1.0" 202 Accepted 2026-03-29 16:58:56,982 - lava_callback - INFO - Starting async job submit for node 69c6b74002375b4506dfb404 2026-03-29 16:58:56,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:57,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b74002375b4506dfb404 HTTP/1.1" 200 None 2026-03-29 16:58:57,006 - lava_callback - DEBUG - Processing node 69c6b74002375b4506dfb404: baseline-arm64-baylibre 2026-03-29 16:58:57,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b74002375b4506dfb404 2026-03-29 16:58:57,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:57,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b74002375b4506dfb404 HTTP/1.1" 200 None 2026-03-29 16:58:57,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:57,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 16:58:57,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:58:57,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b74002375b4506dfb404 HTTP/1.1" 200 None 2026-03-29 16:58:57,103 - lava_callback - ERROR - Error processing callback for node 69c6b74002375b4506dfb404: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 16:59:00,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 16:59:00,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49892 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 17:00:15,818 - lava_callback - INFO - Received callback for node 69c958d4f2866272ea51e2f6 INFO: 172.20.0.1:49908 - "POST /node/69c958d4f2866272ea51e2f6 HTTP/1.0" 202 Accepted 2026-03-29 17:00:15,832 - lava_callback - INFO - Starting async job submit for node 69c958d4f2866272ea51e2f6 2026-03-29 17:00:15,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:15,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d4f2866272ea51e2f6 HTTP/1.1" 200 None 2026-03-29 17:00:15,925 - lava_callback - DEBUG - Processing node 69c958d4f2866272ea51e2f6: baseline-arm64 2026-03-29 17:00:16,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49924 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:00:18,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:00:18,145 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d4f2866272ea51e2f6/log.txt.gz 2026-03-29 17:00:18,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:00:18,845 - lava_callback - INFO - Received callback for node 69c958d6f2866272ea51e2fc INFO: 172.20.0.1:49940 - "POST /node/69c958d6f2866272ea51e2fc HTTP/1.0" 202 Accepted 2026-03-29 17:00:18,854 - lava_callback - INFO - Starting async job submit for node 69c958d6f2866272ea51e2fc 2026-03-29 17:00:18,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:18,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d6f2866272ea51e2fc HTTP/1.1" 200 None 2026-03-29 17:00:18,905 - lava_callback - DEBUG - Processing node 69c958d6f2866272ea51e2fc: baseline-nfs-arm64 2026-03-29 17:00:19,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:00:20,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:00:20,355 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d4f2866272ea51e2f6/lava-logs.yaml 2026-03-29 17:00:20,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:00:21,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:00:21,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d6f2866272ea51e2fc/log.txt.gz 2026-03-29 17:00:21,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:00:22,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:00:22,035 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958d4f2866272ea51e2f6/lava_callback.json.gz DEBUG: Setup failed for 69c958d4f2866272ea51e2f6. Transitting job node result: incomplete -> fail 2026-03-29 17:00:22,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:22,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d4f2866272ea51e2f6 HTTP/1.1" 200 None 2026-03-29 17:00:22,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:22,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:00:22,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:22,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958d4f2866272ea51e2f6 HTTP/1.1" 200 None 2026-03-29 17:00:22,352 - lava_callback - INFO - Completed processing callback for node 69c958d4f2866272ea51e2f6 2026-03-29 17:00:23,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:00:23,484 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d6f2866272ea51e2fc/lava-logs.yaml 2026-03-29 17:00:23,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:00:25,139 - lava_callback - INFO - Received callback for node 69c6b79502375b4506dfb416 INFO: 172.20.0.1:37084 - "POST /node/69c6b79502375b4506dfb416 HTTP/1.0" 202 Accepted 2026-03-29 17:00:25,148 - lava_callback - INFO - Starting async job submit for node 69c6b79502375b4506dfb416 2026-03-29 17:00:25,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:25,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b79502375b4506dfb416 HTTP/1.1" 200 None 2026-03-29 17:00:25,174 - lava_callback - DEBUG - Processing node 69c6b79502375b4506dfb416: baseline-arm64-clabbe 2026-03-29 17:00:25,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b79502375b4506dfb416 2026-03-29 17:00:25,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:25,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b79502375b4506dfb416 HTTP/1.1" 200 None 2026-03-29 17:00:25,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:25,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 17:00:25,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:25,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b79502375b4506dfb416 HTTP/1.1" 200 None 2026-03-29 17:00:25,249 - lava_callback - ERROR - Error processing callback for node 69c6b79502375b4506dfb416: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:00:25,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:00:25,333 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d6f2866272ea51e2fc/lava_callback.json.gz 2026-03-29 17:00:25,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:25,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d6f2866272ea51e2fc HTTP/1.1" 200 None 2026-03-29 17:00:25,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:25,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:00:25,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:25,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958d6f2866272ea51e2fc HTTP/1.1" 200 None 2026-03-29 17:00:25,540 - lava_callback - INFO - Completed processing callback for node 69c958d6f2866272ea51e2fc INFO: 172.20.0.1:37096 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:00:30,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:00:30,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52992 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:01:45,360 - lava_callback - INFO - Received callback for node 69c6b7e302375b4506dfb428 INFO: 172.20.0.1:52326 - "POST /node/69c6b7e302375b4506dfb428 HTTP/1.0" 202 Accepted 2026-03-29 17:01:45,365 - lava_callback - INFO - Starting async job submit for node 69c6b7e302375b4506dfb428 2026-03-29 17:01:45,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:01:45,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b7e302375b4506dfb428 HTTP/1.1" 200 None 2026-03-29 17:01:45,384 - lava_callback - DEBUG - Processing node 69c6b7e302375b4506dfb428: baseline-arm64-clabbe 2026-03-29 17:01:45,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b7e302375b4506dfb428 2026-03-29 17:01:45,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:01:45,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b7e302375b4506dfb428 HTTP/1.1" 200 None 2026-03-29 17:01:45,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:01:45,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 17:01:45,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:01:45,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b7e302375b4506dfb428 HTTP/1.1" 200 None 2026-03-29 17:01:45,439 - lava_callback - ERROR - Error processing callback for node 69c6b7e302375b4506dfb428: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33904 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:02:21,034 - lava_callback - INFO - Received callback for node 69c95681f2866272ea51e0b0 INFO: 172.20.0.1:54100 - "POST /node/69c95681f2866272ea51e0b0 HTTP/1.0" 202 Accepted 2026-03-29 17:02:21,081 - lava_callback - INFO - Starting async job submit for node 69c95681f2866272ea51e0b0 2026-03-29 17:02:21,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:21,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95681f2866272ea51e0b0 HTTP/1.1" 200 None 2026-03-29 17:02:21,122 - lava_callback - DEBUG - Processing node 69c95681f2866272ea51e0b0: baseline-arm-pengutronix 2026-03-29 17:02:21,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:02:21,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:02:21,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95681f2866272ea51e0b0/log.txt.gz 2026-03-29 17:02:21,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:02:21,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:02:21,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95681f2866272ea51e0b0/lava-logs.yaml 2026-03-29 17:02:21,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:02:21,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:02:21,946 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c95681f2866272ea51e0b0/lava_callback.json.gz DEBUG: Setup failed for 69c95681f2866272ea51e0b0. Transitting job node result: incomplete -> fail 2026-03-29 17:02:21,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:22,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95681f2866272ea51e0b0 HTTP/1.1" 200 None 2026-03-29 17:02:22,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:22,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 17:02:22,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:22,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95681f2866272ea51e0b0 HTTP/1.1" 200 None 2026-03-29 17:02:22,162 - lava_callback - INFO - Completed processing callback for node 69c95681f2866272ea51e0b0 INFO: 172.20.0.1:54108 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:02:30,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:30,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40506 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:02:56,560 - lava_callback - INFO - Received callback for node 69c95b27f2866272ea51e5b7 INFO: 172.20.0.1:40522 - "POST /node/69c95b27f2866272ea51e5b7 HTTP/1.0" 202 Accepted 2026-03-29 17:02:56,574 - lava_callback - INFO - Starting async job submit for node 69c95b27f2866272ea51e5b7 2026-03-29 17:02:56,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:56,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b27f2866272ea51e5b7 HTTP/1.1" 200 None 2026-03-29 17:02:56,609 - lava_callback - DEBUG - Processing node 69c95b27f2866272ea51e5b7: baseline-arm 2026-03-29 17:02:56,617 - lava_callback - WARNING - Failed to upload log 2026-03-29 17:02:56,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:02:56,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:02:56,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c95b27f2866272ea51e5b7/lava-logs.yaml 2026-03-29 17:02:56,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:02:56,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:02:56,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c95b27f2866272ea51e5b7/lava_callback.json.gz 2026-03-29 17:02:56,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:56,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b27f2866272ea51e5b7 HTTP/1.1" 200 None 2026-03-29 17:02:56,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:56,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 17:02:56,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:56,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95b27f2866272ea51e5b7 HTTP/1.1" 200 None 2026-03-29 17:02:56,949 - lava_callback - INFO - Completed processing callback for node 69c95b27f2866272ea51e5b7 2026-03-29 17:02:57,094 - lava_callback - INFO - Received callback for node 69c6b83d02375b4506dfb493 INFO: 172.20.0.1:40538 - "POST /node/69c6b83d02375b4506dfb493 HTTP/1.0" 202 Accepted 2026-03-29 17:02:57,101 - lava_callback - INFO - Starting async job submit for node 69c6b83d02375b4506dfb493 2026-03-29 17:02:57,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b83d02375b4506dfb493 HTTP/1.1" 200 None 2026-03-29 17:02:57,127 - lava_callback - DEBUG - Processing node 69c6b83d02375b4506dfb493: baseline-arm-baylibre 2026-03-29 17:02:57,128 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b83d02375b4506dfb493 2026-03-29 17:02:57,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b83d02375b4506dfb493 HTTP/1.1" 200 None 2026-03-29 17:02:57,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:02:57,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b83d02375b4506dfb493 HTTP/1.1" 200 None 2026-03-29 17:02:57,186 - lava_callback - ERROR - Error processing callback for node 69c6b83d02375b4506dfb493: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:02:57,777 - lava_callback - INFO - Received callback for node 69c6b83e02375b4506dfb496 INFO: 172.20.0.1:40548 - "POST /node/69c6b83e02375b4506dfb496 HTTP/1.0" 202 Accepted 2026-03-29 17:02:57,784 - lava_callback - INFO - Starting async job submit for node 69c6b83e02375b4506dfb496 2026-03-29 17:02:57,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b83e02375b4506dfb496 HTTP/1.1" 200 None 2026-03-29 17:02:57,801 - lava_callback - DEBUG - Processing node 69c6b83e02375b4506dfb496: baseline-arm-baylibre 2026-03-29 17:02:57,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b83e02375b4506dfb496 2026-03-29 17:02:57,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b83e02375b4506dfb496 HTTP/1.1" 200 None 2026-03-29 17:02:57,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:02:57,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:02:57,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b83e02375b4506dfb496 HTTP/1.1" 200 None 2026-03-29 17:02:57,883 - lava_callback - ERROR - Error processing callback for node 69c6b83e02375b4506dfb496: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:00,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:00,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 17:03:05,238 - lava_callback - INFO - Received callback for node 69c6b83302375b4506dfb463 INFO: 172.20.0.1:60850 - "POST /node/69c6b83302375b4506dfb463 HTTP/1.0" 202 Accepted 2026-03-29 17:03:05,244 - lava_callback - INFO - Starting async job submit for node 69c6b83302375b4506dfb463 2026-03-29 17:03:05,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:05,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b83302375b4506dfb463 HTTP/1.1" 200 None 2026-03-29 17:03:05,260 - lava_callback - DEBUG - Processing node 69c6b83302375b4506dfb463: baseline-arm64-clabbe 2026-03-29 17:03:05,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b83302375b4506dfb463 2026-03-29 17:03:05,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:05,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b83302375b4506dfb463 HTTP/1.1" 200 None 2026-03-29 17:03:05,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:05,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 17:03:05,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:05,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b83302375b4506dfb463 HTTP/1.1" 200 None 2026-03-29 17:03:05,336 - lava_callback - ERROR - Error processing callback for node 69c6b83302375b4506dfb463: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41726 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:03:17,077 - lava_callback - INFO - Received callback for node 69c6b84002375b4506dfb499 INFO: 172.20.0.1:41740 - "POST /node/69c6b84002375b4506dfb499 HTTP/1.0" 202 Accepted 2026-03-29 17:03:17,083 - lava_callback - INFO - Starting async job submit for node 69c6b84002375b4506dfb499 2026-03-29 17:03:17,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84002375b4506dfb499 HTTP/1.1" 200 None 2026-03-29 17:03:17,107 - lava_callback - DEBUG - Processing node 69c6b84002375b4506dfb499: baseline-arm-baylibre 2026-03-29 17:03:17,107 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b84002375b4506dfb499 2026-03-29 17:03:17,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84002375b4506dfb499 HTTP/1.1" 200 None 2026-03-29 17:03:17,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:17,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b84002375b4506dfb499 HTTP/1.1" 200 None 2026-03-29 17:03:17,167 - lava_callback - ERROR - Error processing callback for node 69c6b84002375b4506dfb499: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:17,734 - lava_callback - INFO - Received callback for node 69c6b84102375b4506dfb49d INFO: 172.20.0.1:41752 - "POST /node/69c6b84102375b4506dfb49d HTTP/1.0" 202 Accepted 2026-03-29 17:03:17,740 - lava_callback - INFO - Starting async job submit for node 69c6b84102375b4506dfb49d 2026-03-29 17:03:17,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84102375b4506dfb49d HTTP/1.1" 200 None 2026-03-29 17:03:17,757 - lava_callback - DEBUG - Processing node 69c6b84102375b4506dfb49d: baseline-arm-baylibre 2026-03-29 17:03:17,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b84102375b4506dfb49d 2026-03-29 17:03:17,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84102375b4506dfb49d HTTP/1.1" 200 None 2026-03-29 17:03:17,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:17,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:17,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b84102375b4506dfb49d HTTP/1.1" 200 None 2026-03-29 17:03:17,818 - lava_callback - ERROR - Error processing callback for node 69c6b84102375b4506dfb49d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:18,410 - lava_callback - INFO - Received callback for node 69c6b84402375b4506dfb4b0 INFO: 172.20.0.1:41754 - "POST /node/69c6b84402375b4506dfb4b0 HTTP/1.0" 202 Accepted 2026-03-29 17:03:18,417 - lava_callback - INFO - Starting async job submit for node 69c6b84402375b4506dfb4b0 2026-03-29 17:03:18,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:18,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84402375b4506dfb4b0 HTTP/1.1" 200 None 2026-03-29 17:03:18,432 - lava_callback - DEBUG - Processing node 69c6b84402375b4506dfb4b0: baseline-arm-baylibre 2026-03-29 17:03:18,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b84402375b4506dfb4b0 2026-03-29 17:03:18,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:18,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84402375b4506dfb4b0 HTTP/1.1" 200 None 2026-03-29 17:03:18,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:18,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:18,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:18,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b84402375b4506dfb4b0 HTTP/1.1" 200 None 2026-03-29 17:03:18,502 - lava_callback - ERROR - Error processing callback for node 69c6b84402375b4506dfb4b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:19,108 - lava_callback - INFO - Received callback for node 69c6b84502375b4506dfb4b3 INFO: 172.20.0.1:41762 - "POST /node/69c6b84502375b4506dfb4b3 HTTP/1.0" 202 Accepted 2026-03-29 17:03:19,114 - lava_callback - INFO - Starting async job submit for node 69c6b84502375b4506dfb4b3 2026-03-29 17:03:19,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:19,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84502375b4506dfb4b3 HTTP/1.1" 200 None 2026-03-29 17:03:19,130 - lava_callback - DEBUG - Processing node 69c6b84502375b4506dfb4b3: baseline-arm-baylibre 2026-03-29 17:03:19,130 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b84502375b4506dfb4b3 2026-03-29 17:03:19,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:19,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84502375b4506dfb4b3 HTTP/1.1" 200 None 2026-03-29 17:03:19,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:19,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:19,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:19,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b84502375b4506dfb4b3 HTTP/1.1" 200 None 2026-03-29 17:03:19,182 - lava_callback - ERROR - Error processing callback for node 69c6b84502375b4506dfb4b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:25,243 - lava_callback - INFO - Received callback for node 69c6b84d02375b4506dfb4ca INFO: 172.20.0.1:47114 - "POST /node/69c6b84d02375b4506dfb4ca HTTP/1.0" 202 Accepted 2026-03-29 17:03:25,248 - lava_callback - INFO - Starting async job submit for node 69c6b84d02375b4506dfb4ca 2026-03-29 17:03:25,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:25,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84d02375b4506dfb4ca HTTP/1.1" 200 None 2026-03-29 17:03:25,265 - lava_callback - DEBUG - Processing node 69c6b84d02375b4506dfb4ca: baseline-arm-clabbe 2026-03-29 17:03:25,265 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b84d02375b4506dfb4ca 2026-03-29 17:03:25,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:25,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b84d02375b4506dfb4ca HTTP/1.1" 200 None 2026-03-29 17:03:25,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:25,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:25,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:25,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b84d02375b4506dfb4ca HTTP/1.1" 200 None 2026-03-29 17:03:25,333 - lava_callback - ERROR - Error processing callback for node 69c6b84d02375b4506dfb4ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:25,918 - lava_callback - INFO - Received callback for node 69c6b85102375b4506dfb4d7 INFO: 172.20.0.1:47130 - "POST /node/69c6b85102375b4506dfb4d7 HTTP/1.0" 202 Accepted 2026-03-29 17:03:25,926 - lava_callback - INFO - Starting async job submit for node 69c6b85102375b4506dfb4d7 2026-03-29 17:03:25,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:25,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85102375b4506dfb4d7 HTTP/1.1" 200 None 2026-03-29 17:03:25,948 - lava_callback - DEBUG - Processing node 69c6b85102375b4506dfb4d7: baseline-arm-clabbe 2026-03-29 17:03:25,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b85102375b4506dfb4d7 2026-03-29 17:03:25,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:25,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85102375b4506dfb4d7 HTTP/1.1" 200 None 2026-03-29 17:03:25,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:25,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:25,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:26,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b85102375b4506dfb4d7 HTTP/1.1" 200 None 2026-03-29 17:03:26,010 - lava_callback - ERROR - Error processing callback for node 69c6b85102375b4506dfb4d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:03:26,597 - lava_callback - INFO - Received callback for node 69c6b85202375b4506dfb4da INFO: 172.20.0.1:47144 - "POST /node/69c6b85202375b4506dfb4da HTTP/1.0" 202 Accepted 2026-03-29 17:03:26,603 - lava_callback - INFO - Starting async job submit for node 69c6b85202375b4506dfb4da 2026-03-29 17:03:26,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:26,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85202375b4506dfb4da HTTP/1.1" 200 None 2026-03-29 17:03:26,619 - lava_callback - DEBUG - Processing node 69c6b85202375b4506dfb4da: baseline-arm-clabbe 2026-03-29 17:03:26,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b85202375b4506dfb4da 2026-03-29 17:03:26,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:26,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85202375b4506dfb4da HTTP/1.1" 200 None 2026-03-29 17:03:26,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:26,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:26,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:26,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b85202375b4506dfb4da HTTP/1.1" 200 None 2026-03-29 17:03:26,670 - lava_callback - ERROR - Error processing callback for node 69c6b85202375b4506dfb4da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:27,299 - lava_callback - INFO - Received callback for node 69c6b85302375b4506dfb4dd INFO: 172.20.0.1:47156 - "POST /node/69c6b85302375b4506dfb4dd HTTP/1.0" 202 Accepted 2026-03-29 17:03:27,304 - lava_callback - INFO - Starting async job submit for node 69c6b85302375b4506dfb4dd 2026-03-29 17:03:27,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:27,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85302375b4506dfb4dd HTTP/1.1" 200 None 2026-03-29 17:03:27,321 - lava_callback - DEBUG - Processing node 69c6b85302375b4506dfb4dd: baseline-arm-clabbe 2026-03-29 17:03:27,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b85302375b4506dfb4dd 2026-03-29 17:03:27,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:27,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85302375b4506dfb4dd HTTP/1.1" 200 None 2026-03-29 17:03:27,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:27,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:27,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:27,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b85302375b4506dfb4dd HTTP/1.1" 200 None 2026-03-29 17:03:27,383 - lava_callback - ERROR - Error processing callback for node 69c6b85302375b4506dfb4dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:27,999 - lava_callback - INFO - Received callback for node 69c6b85502375b4506dfb4e6 INFO: 172.20.0.1:47164 - "POST /node/69c6b85502375b4506dfb4e6 HTTP/1.0" 202 Accepted 2026-03-29 17:03:28,008 - lava_callback - INFO - Starting async job submit for node 69c6b85502375b4506dfb4e6 2026-03-29 17:03:28,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:28,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85502375b4506dfb4e6 HTTP/1.1" 200 None 2026-03-29 17:03:28,025 - lava_callback - DEBUG - Processing node 69c6b85502375b4506dfb4e6: baseline-arm-clabbe 2026-03-29 17:03:28,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b85502375b4506dfb4e6 2026-03-29 17:03:28,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:28,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b85502375b4506dfb4e6 HTTP/1.1" 200 None 2026-03-29 17:03:28,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:28,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 17:03:28,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:28,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b85502375b4506dfb4e6 HTTP/1.1" 200 None 2026-03-29 17:03:28,086 - lava_callback - ERROR - Error processing callback for node 69c6b85502375b4506dfb4e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:03:34,277 - lava_callback - INFO - Received callback for node 69c958cdf2866272ea51e2e6 INFO: 172.20.0.1:41268 - "POST /node/69c958cdf2866272ea51e2e6 HTTP/1.0" 202 Accepted 2026-03-29 17:03:34,283 - lava_callback - INFO - Starting async job submit for node 69c958cdf2866272ea51e2e6 2026-03-29 17:03:34,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:34,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958cdf2866272ea51e2e6 HTTP/1.1" 200 None 2026-03-29 17:03:34,332 - lava_callback - DEBUG - Processing node 69c958cdf2866272ea51e2e6: baseline-arm64 2026-03-29 17:03:34,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:03:35,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:03:35,014 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cdf2866272ea51e2e6/log.txt.gz 2026-03-29 17:03:35,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:03:35,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:03:35,507 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cdf2866272ea51e2e6/lava-logs.yaml 2026-03-29 17:03:35,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:03:35,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:03:35,692 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c958cdf2866272ea51e2e6/lava_callback.json.gz 2026-03-29 17:03:35,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:35,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958cdf2866272ea51e2e6 HTTP/1.1" 200 None 2026-03-29 17:03:35,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:35,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:03:35,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:03:35,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958cdf2866272ea51e2e6 HTTP/1.1" 200 None 2026-03-29 17:03:35,833 - lava_callback - INFO - Completed processing callback for node 69c958cdf2866272ea51e2e6 INFO: 172.20.0.1:41284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57940 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:04:00,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:00,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 17:04:05,306 - lava_callback - INFO - Received callback for node 69c6b88102375b4506dfb57d INFO: 172.20.0.1:44620 - "POST /node/69c6b88102375b4506dfb57d HTTP/1.0" 202 Accepted 2026-03-29 17:04:05,311 - lava_callback - INFO - Starting async job submit for node 69c6b88102375b4506dfb57d 2026-03-29 17:04:05,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:05,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b88102375b4506dfb57d HTTP/1.1" 200 None 2026-03-29 17:04:05,337 - lava_callback - DEBUG - Processing node 69c6b88102375b4506dfb57d: baseline-arm64-clabbe 2026-03-29 17:04:05,338 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b88102375b4506dfb57d 2026-03-29 17:04:05,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:05,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b88102375b4506dfb57d HTTP/1.1" 200 None 2026-03-29 17:04:05,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:05,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 17:04:05,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:05,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b88102375b4506dfb57d HTTP/1.1" 200 None 2026-03-29 17:04:05,394 - lava_callback - ERROR - Error processing callback for node 69c6b88102375b4506dfb57d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34074 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:04:14,745 - lava_callback - INFO - Received callback for node 69c95b79f2866272ea51e5e7 INFO: 172.20.0.1:34080 - "POST /node/69c95b79f2866272ea51e5e7 HTTP/1.0" 202 Accepted 2026-03-29 17:04:14,751 - lava_callback - INFO - Starting async job submit for node 69c95b79f2866272ea51e5e7 2026-03-29 17:04:14,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:14,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b79f2866272ea51e5e7 HTTP/1.1" 200 None 2026-03-29 17:04:14,769 - lava_callback - DEBUG - Processing node 69c95b79f2866272ea51e5e7: baseline-nfs-arm64 2026-03-29 17:04:14,771 - lava_callback - WARNING - Failed to upload log 2026-03-29 17:04:14,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:04:14,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:04:14,901 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c95b79f2866272ea51e5e7/lava-logs.yaml 2026-03-29 17:04:14,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:04:15,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:04:15,020 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c95b79f2866272ea51e5e7/lava_callback.json.gz 2026-03-29 17:04:15,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:15,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b79f2866272ea51e5e7 HTTP/1.1" 200 None 2026-03-29 17:04:15,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:15,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:04:15,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:15,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95b79f2866272ea51e5e7 HTTP/1.1" 200 None 2026-03-29 17:04:15,087 - lava_callback - INFO - Completed processing callback for node 69c95b79f2866272ea51e5e7 INFO: 172.20.0.1:34086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37990 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:04:30,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:04:30,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58744 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:05:25,330 - lava_callback - INFO - Received callback for node 69c6b8cf02375b4506dfb5dd INFO: 172.20.0.1:52020 - "POST /node/69c6b8cf02375b4506dfb5dd HTTP/1.0" 202 Accepted 2026-03-29 17:05:25,335 - lava_callback - INFO - Starting async job submit for node 69c6b8cf02375b4506dfb5dd 2026-03-29 17:05:25,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:05:25,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b8cf02375b4506dfb5dd HTTP/1.1" 200 None 2026-03-29 17:05:25,351 - lava_callback - DEBUG - Processing node 69c6b8cf02375b4506dfb5dd: baseline-arm64-clabbe 2026-03-29 17:05:25,351 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b8cf02375b4506dfb5dd 2026-03-29 17:05:25,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:05:25,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b8cf02375b4506dfb5dd HTTP/1.1" 200 None 2026-03-29 17:05:25,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:05:25,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 17:05:25,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:05:25,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b8cf02375b4506dfb5dd HTTP/1.1" 200 None 2026-03-29 17:05:25,426 - lava_callback - ERROR - Error processing callback for node 69c6b8cf02375b4506dfb5dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39112 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:06:45,406 - lava_callback - INFO - Received callback for node 69c6b91d02375b4506dfb5fc INFO: 172.20.0.1:52316 - "POST /node/69c6b91d02375b4506dfb5fc HTTP/1.0" 202 Accepted 2026-03-29 17:06:45,411 - lava_callback - INFO - Starting async job submit for node 69c6b91d02375b4506dfb5fc 2026-03-29 17:06:45,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:06:45,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b91d02375b4506dfb5fc HTTP/1.1" 200 None 2026-03-29 17:06:45,429 - lava_callback - DEBUG - Processing node 69c6b91d02375b4506dfb5fc: baseline-arm64-clabbe 2026-03-29 17:06:45,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b91d02375b4506dfb5fc 2026-03-29 17:06:45,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:06:45,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b91d02375b4506dfb5fc HTTP/1.1" 200 None 2026-03-29 17:06:45,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:06:45,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 17:06:45,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:06:45,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b91d02375b4506dfb5fc HTTP/1.1" 200 None 2026-03-29 17:06:45,485 - lava_callback - ERROR - Error processing callback for node 69c6b91d02375b4506dfb5fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48040 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:07:45,364 - lava_callback - INFO - Received callback for node 69c6b95e02375b4506dfb613 INFO: 172.20.0.1:49846 - "POST /node/69c6b95e02375b4506dfb613 HTTP/1.0" 202 Accepted 2026-03-29 17:07:45,370 - lava_callback - INFO - Starting async job submit for node 69c6b95e02375b4506dfb613 2026-03-29 17:07:45,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:07:45,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b95e02375b4506dfb613 HTTP/1.1" 200 None 2026-03-29 17:07:45,391 - lava_callback - DEBUG - Processing node 69c6b95e02375b4506dfb613: baseline-arm-clabbe 2026-03-29 17:07:45,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6b95e02375b4506dfb613 2026-03-29 17:07:45,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:07:45,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6b95e02375b4506dfb613 HTTP/1.1" 200 None 2026-03-29 17:07:45,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:07:45,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 17:07:45,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:07:45,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6b95e02375b4506dfb613 HTTP/1.1" 200 None 2026-03-29 17:07:45,458 - lava_callback - ERROR - Error processing callback for node 69c6b95e02375b4506dfb613: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38400 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:09:00,704 - lava_callback - INFO - Received callback for node 69c95c76f2866272ea51e629 INFO: 172.20.0.1:47154 - "POST /node/69c95c76f2866272ea51e629 HTTP/1.0" 202 Accepted 2026-03-29 17:09:00,710 - lava_callback - INFO - Starting async job submit for node 69c95c76f2866272ea51e629 2026-03-29 17:09:00,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:09:00,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95c76f2866272ea51e629 HTTP/1.1" 200 None 2026-03-29 17:09:00,759 - lava_callback - DEBUG - Processing node 69c95c76f2866272ea51e629: baseline-arm64 2026-03-29 17:09:00,900 - lava_callback - WARNING - Failed to upload log 2026-03-29 17:09:00,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:09:01,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:09:01,346 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c95c76f2866272ea51e629/lava-logs.yaml 2026-03-29 17:09:01,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:09:01,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:09:01,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c95c76f2866272ea51e629/lava_callback.json.gz 2026-03-29 17:09:01,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:09:01,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95c76f2866272ea51e629 HTTP/1.1" 200 None 2026-03-29 17:09:01,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:09:01,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:09:01,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:09:01,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95c76f2866272ea51e629 HTTP/1.1" 200 None 2026-03-29 17:09:01,648 - lava_callback - INFO - Completed processing callback for node 69c95c76f2866272ea51e629 INFO: 172.20.0.1:47164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55138 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:09:30,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:09:30,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40930 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:10:25,493 - lava_callback - INFO - Received callback for node 69c95cc2f2866272ea51e633 INFO: 172.20.0.1:55120 - "POST /node/69c95cc2f2866272ea51e633 HTTP/1.0" 202 Accepted 2026-03-29 17:10:25,500 - lava_callback - INFO - Starting async job submit for node 69c95cc2f2866272ea51e633 2026-03-29 17:10:25,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:10:25,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95cc2f2866272ea51e633 HTTP/1.1" 200 None 2026-03-29 17:10:25,545 - lava_callback - DEBUG - Processing node 69c95cc2f2866272ea51e633: baseline-arm64 2026-03-29 17:10:25,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:10:25,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:10:25,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c95cc2f2866272ea51e633/log.txt.gz 2026-03-29 17:10:25,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:10:26,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:10:26,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c95cc2f2866272ea51e633/lava-logs.yaml 2026-03-29 17:10:26,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:10:26,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:10:26,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c95cc2f2866272ea51e633/lava_callback.json.gz 2026-03-29 17:10:26,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:10:26,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95cc2f2866272ea51e633 HTTP/1.1" 200 None 2026-03-29 17:10:26,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:10:26,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:10:26,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:10:26,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95cc2f2866272ea51e633 HTTP/1.1" 200 None INFO: 172.20.0.1:55122 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:10:26,587 - lava_callback - INFO - Completed processing callback for node 69c95cc2f2866272ea51e633 2026-03-29 17:10:30,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:10:30,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39086 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:12:45,066 - lava_callback - INFO - Received callback for node 69c958d9f2866272ea51e302 INFO: 172.20.0.1:56790 - "POST /node/69c958d9f2866272ea51e302 HTTP/1.0" 202 Accepted 2026-03-29 17:12:45,075 - lava_callback - INFO - Starting async job submit for node 69c958d9f2866272ea51e302 2026-03-29 17:12:45,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:12:45,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d9f2866272ea51e302 HTTP/1.1" 200 None 2026-03-29 17:12:45,134 - lava_callback - DEBUG - Processing node 69c958d9f2866272ea51e302: baseline-nfs-arm64 2026-03-29 17:12:46,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56792 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:12:46,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:12:46,646 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d9f2866272ea51e302/log.txt.gz 2026-03-29 17:12:47,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:12:47,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:12:47,570 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d9f2866272ea51e302/lava-logs.yaml 2026-03-29 17:12:47,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:12:47,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:12:47,770 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958d9f2866272ea51e302/lava_callback.json.gz DEBUG: Setup failed for 69c958d9f2866272ea51e302. Transitting job node result: incomplete -> fail 2026-03-29 17:12:47,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:12:47,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958d9f2866272ea51e302 HTTP/1.1" 200 None 2026-03-29 17:12:47,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:12:47,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:12:47,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:12:47,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958d9f2866272ea51e302 HTTP/1.1" 200 None 2026-03-29 17:12:48,016 - lava_callback - INFO - Completed processing callback for node 69c958d9f2866272ea51e302 INFO: 172.20.0.1:59420 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:13:00,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:13:00,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59144 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:14:08,277 - lava_callback - INFO - Received callback for node 69c956a9f2866272ea51e116 INFO: 172.20.0.1:59150 - "POST /node/69c956a9f2866272ea51e116 HTTP/1.0" 202 Accepted 2026-03-29 17:14:08,325 - lava_callback - INFO - Starting async job submit for node 69c956a9f2866272ea51e116 2026-03-29 17:14:08,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:14:08,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a9f2866272ea51e116 HTTP/1.1" 200 None 2026-03-29 17:14:08,390 - lava_callback - DEBUG - Processing node 69c956a9f2866272ea51e116: ltp-timers 2026-03-29 17:14:08,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:14:08,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:14:08,787 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c956a9f2866272ea51e116/log.txt.gz 2026-03-29 17:14:08,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:14:09,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:14:09,021 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c956a9f2866272ea51e116/lava-logs.yaml 2026-03-29 17:14:09,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:14:09,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:14:09,317 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c956a9f2866272ea51e116/lava_callback.json.gz 2026-03-29 17:14:09,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:14:09,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c956a9f2866272ea51e116 HTTP/1.1" 200 None 2026-03-29 17:14:09,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:14:09,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 17:14:09,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:14:09,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c956a9f2866272ea51e116 HTTP/1.1" 200 None 2026-03-29 17:14:09,448 - lava_callback - INFO - Completed processing callback for node 69c956a9f2866272ea51e116 INFO: 172.20.0.1:59578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50776 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:14:30,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:14:30,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39592 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:16:36,169 - lava_callback - INFO - Received callback for node 69c958e0f2866272ea51e315 INFO: 172.20.0.1:41622 - "POST /node/69c958e0f2866272ea51e315 HTTP/1.0" 202 Accepted 2026-03-29 17:16:36,176 - lava_callback - INFO - Starting async job submit for node 69c958e0f2866272ea51e315 2026-03-29 17:16:36,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:16:36,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e0f2866272ea51e315 HTTP/1.1" 200 None 2026-03-29 17:16:36,218 - lava_callback - DEBUG - Processing node 69c958e0f2866272ea51e315: baseline-nfs-arm64 INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:16:36,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:16:37,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:16:37,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e0f2866272ea51e315/log.txt.gz 2026-03-29 17:16:37,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:16:37,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:16:37,603 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e0f2866272ea51e315/lava-logs.yaml 2026-03-29 17:16:37,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:16:37,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:16:37,789 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c958e0f2866272ea51e315/lava_callback.json.gz 2026-03-29 17:16:37,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:16:37,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e0f2866272ea51e315 HTTP/1.1" 200 None 2026-03-29 17:16:37,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:16:37,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:16:37,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:16:37,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958e0f2866272ea51e315 HTTP/1.1" 200 None 2026-03-29 17:16:37,928 - lava_callback - INFO - Completed processing callback for node 69c958e0f2866272ea51e315 INFO: 172.20.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44930 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:17:00,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:17:00,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57284 - "GET /node/69865d35177ca0f097bd45fc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56426 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:19:59,809 - lava_callback - INFO - Received callback for node 69c95b26f2866272ea51e5b4 INFO: 172.20.0.1:40760 - "POST /node/69c95b26f2866272ea51e5b4 HTTP/1.0" 202 Accepted 2026-03-29 17:19:59,816 - lava_callback - INFO - Starting async job submit for node 69c95b26f2866272ea51e5b4 2026-03-29 17:19:59,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:19:59,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b26f2866272ea51e5b4 HTTP/1.1" 200 None 2026-03-29 17:19:59,890 - lava_callback - DEBUG - Processing node 69c95b26f2866272ea51e5b4: ltp-smoke 2026-03-29 17:20:00,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:20:00,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:20:00,565 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c95b26f2866272ea51e5b4/log.txt.gz 2026-03-29 17:20:00,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:20:01,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:20:01,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c95b26f2866272ea51e5b4/lava-logs.yaml 2026-03-29 17:20:01,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:20:01,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:20:01,417 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c95b26f2866272ea51e5b4/lava_callback.json.gz 2026-03-29 17:20:01,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:20:01,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b26f2866272ea51e5b4 HTTP/1.1" 200 None 2026-03-29 17:20:01,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:20:01,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950aef2866272ea51b8ca HTTP/1.1" 200 None 2026-03-29 17:20:01,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:20:01,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95b26f2866272ea51e5b4 HTTP/1.1" 200 None 2026-03-29 17:20:01,740 - lava_callback - INFO - Completed processing callback for node 69c95b26f2866272ea51e5b4 INFO: 172.20.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36718 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:20:30,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:20:30,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50954 - "GET /node/69894ad085773790cd34e1f6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57080 - "GET /node/6949e7952060d3d84791d1a2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57086 - "GET /node/698bd76f1e517752bab8b49c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53012 - "GET /node/698af0243b3beb19e76161cc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37684 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:29:18,697 - lava_callback - INFO - Received callback for node 69c6be6802375b4506dfb7ba INFO: 172.20.0.1:37692 - "POST /node/69c6be6802375b4506dfb7ba HTTP/1.0" 202 Accepted 2026-03-29 17:29:18,703 - lava_callback - INFO - Starting async job submit for node 69c6be6802375b4506dfb7ba 2026-03-29 17:29:18,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:29:18,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6be6802375b4506dfb7ba HTTP/1.1" 200 None 2026-03-29 17:29:18,719 - lava_callback - DEBUG - Processing node 69c6be6802375b4506dfb7ba: baseline-arm64-baylibre 2026-03-29 17:29:18,719 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6be6802375b4506dfb7ba 2026-03-29 17:29:18,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:29:18,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6be6802375b4506dfb7ba HTTP/1.1" 200 None 2026-03-29 17:29:18,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:29:18,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:29:18,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:29:18,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6be6802375b4506dfb7ba HTTP/1.1" 200 None 2026-03-29 17:29:18,774 - lava_callback - ERROR - Error processing callback for node 69c6be6802375b4506dfb7ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53262 - "GET /node/69896efb85773790cd34f175%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37916 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:33:18,776 - lava_callback - INFO - Received callback for node 69c6bf4f02375b4506dfb7ef INFO: 172.20.0.1:37924 - "POST /node/69c6bf4f02375b4506dfb7ef HTTP/1.0" 202 Accepted 2026-03-29 17:33:18,782 - lava_callback - INFO - Starting async job submit for node 69c6bf4f02375b4506dfb7ef 2026-03-29 17:33:18,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:33:18,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6bf4f02375b4506dfb7ef HTTP/1.1" 200 None 2026-03-29 17:33:18,799 - lava_callback - DEBUG - Processing node 69c6bf4f02375b4506dfb7ef: baseline-arm64-baylibre 2026-03-29 17:33:18,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6bf4f02375b4506dfb7ef 2026-03-29 17:33:18,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:33:18,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6bf4f02375b4506dfb7ef HTTP/1.1" 200 None 2026-03-29 17:33:18,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:33:18,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:33:18,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:33:18,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6bf4f02375b4506dfb7ef HTTP/1.1" 200 None 2026-03-29 17:33:18,859 - lava_callback - ERROR - Error processing callback for node 69c6bf4f02375b4506dfb7ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41496 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:37:06,879 - lava_callback - INFO - Received callback for node 69c6c03c02375b4506dfb817 INFO: 172.20.0.1:41506 - "POST /node/69c6c03c02375b4506dfb817 HTTP/1.0" 202 Accepted 2026-03-29 17:37:06,884 - lava_callback - INFO - Starting async job submit for node 69c6c03c02375b4506dfb817 2026-03-29 17:37:06,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:06,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c03c02375b4506dfb817 HTTP/1.1" 200 None 2026-03-29 17:37:06,906 - lava_callback - DEBUG - Processing node 69c6c03c02375b4506dfb817: baseline-arm64-clabbe 2026-03-29 17:37:06,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c03c02375b4506dfb817 2026-03-29 17:37:06,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:06,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c03c02375b4506dfb817 HTTP/1.1" 200 None 2026-03-29 17:37:06,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:06,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:37:06,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:06,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c03c02375b4506dfb817 HTTP/1.1" 200 None 2026-03-29 17:37:06,959 - lava_callback - ERROR - Error processing callback for node 69c6c03c02375b4506dfb817: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53018 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:37:37,060 - lava_callback - INFO - Received callback for node 69c9620cf2866272ea51e723 INFO: 172.20.0.1:53034 - "POST /node/69c9620cf2866272ea51e723 HTTP/1.0" 202 Accepted 2026-03-29 17:37:37,070 - lava_callback - INFO - Starting async job submit for node 69c9620cf2866272ea51e723 2026-03-29 17:37:37,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:37,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9620cf2866272ea51e723 HTTP/1.1" 200 None 2026-03-29 17:37:37,143 - lava_callback - DEBUG - Processing node 69c9620cf2866272ea51e723: baseline-arm64-foundriesio 2026-03-29 17:37:38,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:37:38,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:37:38,556 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c9620cf2866272ea51e723/log.txt.gz 2026-03-29 17:37:39,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:37:39,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:37:39,532 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c9620cf2866272ea51e723/lava-logs.yaml 2026-03-29 17:37:39,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:37:39,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:37:39,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c9620cf2866272ea51e723/lava_callback.json.gz DEBUG: Setup failed for 69c9620cf2866272ea51e723. Transitting job node result: incomplete -> fail 2026-03-29 17:37:39,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:39,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9620cf2866272ea51e723 HTTP/1.1" 200 None 2026-03-29 17:37:39,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:39,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:37:39,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:37:39,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9620cf2866272ea51e723 HTTP/1.1" 200 None 2026-03-29 17:37:40,062 - lava_callback - INFO - Completed processing callback for node 69c9620cf2866272ea51e723 INFO: 172.20.0.1:50010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:38:00,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:38:00,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41510 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:39:39,297 - lava_callback - INFO - Received callback for node 69c6c0cb02375b4506dfb831 INFO: 172.20.0.1:41524 - "POST /node/69c6c0cb02375b4506dfb831 HTTP/1.0" 202 Accepted 2026-03-29 17:39:39,303 - lava_callback - INFO - Starting async job submit for node 69c6c0cb02375b4506dfb831 2026-03-29 17:39:39,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:39:39,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c0cb02375b4506dfb831 HTTP/1.1" 200 None 2026-03-29 17:39:39,318 - lava_callback - DEBUG - Processing node 69c6c0cb02375b4506dfb831: baseline-arm64-baylibre 2026-03-29 17:39:39,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c0cb02375b4506dfb831 2026-03-29 17:39:39,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:39:39,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c0cb02375b4506dfb831 HTTP/1.1" 200 None 2026-03-29 17:39:39,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:39:39,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 17:39:39,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:39:39,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c0cb02375b4506dfb831 HTTP/1.1" 200 None 2026-03-29 17:39:39,370 - lava_callback - ERROR - Error processing callback for node 69c6c0cb02375b4506dfb831: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57726 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:41:06,783 - lava_callback - INFO - Received callback for node 69c6c11f02375b4506dfb83e INFO: 172.20.0.1:57730 - "POST /node/69c6c11f02375b4506dfb83e HTTP/1.0" 202 Accepted 2026-03-29 17:41:06,789 - lava_callback - INFO - Starting async job submit for node 69c6c11f02375b4506dfb83e 2026-03-29 17:41:06,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:41:06,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c11f02375b4506dfb83e HTTP/1.1" 200 None 2026-03-29 17:41:06,805 - lava_callback - DEBUG - Processing node 69c6c11f02375b4506dfb83e: baseline-arm64-clabbe 2026-03-29 17:41:06,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c11f02375b4506dfb83e 2026-03-29 17:41:06,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:41:06,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c11f02375b4506dfb83e HTTP/1.1" 200 None 2026-03-29 17:41:06,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:41:06,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:41:06,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:41:06,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c11f02375b4506dfb83e HTTP/1.1" 200 None 2026-03-29 17:41:06,867 - lava_callback - ERROR - Error processing callback for node 69c6c11f02375b4506dfb83e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39312 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:42:11,073 - lava_callback - INFO - Received callback for node 69c962baf2866272ea51e744 INFO: 172.20.0.1:35180 - "POST /node/69c962baf2866272ea51e744 HTTP/1.0" 202 Accepted 2026-03-29 17:42:11,081 - lava_callback - INFO - Starting async job submit for node 69c962baf2866272ea51e744 2026-03-29 17:42:11,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:42:11,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c962baf2866272ea51e744 HTTP/1.1" 200 None 2026-03-29 17:42:11,135 - lava_callback - DEBUG - Processing node 69c962baf2866272ea51e744: baseline-arm64 2026-03-29 17:42:11,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:35196 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:42:11,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:42:11,667 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c962baf2866272ea51e744/log.txt.gz 2026-03-29 17:42:11,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:42:12,133 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:42:12,135 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c962baf2866272ea51e744/lava-logs.yaml 2026-03-29 17:42:12,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:42:12,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:42:12,304 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c962baf2866272ea51e744/lava_callback.json.gz DEBUG: Setup failed for 69c962baf2866272ea51e744. Transitting job node result: incomplete -> fail 2026-03-29 17:42:12,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:42:12,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c962baf2866272ea51e744 HTTP/1.1" 200 None 2026-03-29 17:42:12,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:42:12,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:42:12,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:42:12,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c962baf2866272ea51e744 HTTP/1.1" 200 None 2026-03-29 17:42:12,522 - lava_callback - INFO - Completed processing callback for node 69c962baf2866272ea51e744 INFO: 172.20.0.1:35200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:42:30,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:42:30,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38042 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:44:06,929 - lava_callback - INFO - Received callback for node 69c6c1d002375b4506dfb85f INFO: 172.20.0.1:38052 - "POST /node/69c6c1d002375b4506dfb85f HTTP/1.0" 202 Accepted 2026-03-29 17:44:06,935 - lava_callback - INFO - Starting async job submit for node 69c6c1d002375b4506dfb85f 2026-03-29 17:44:06,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:44:06,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c1d002375b4506dfb85f HTTP/1.1" 200 None 2026-03-29 17:44:06,957 - lava_callback - DEBUG - Processing node 69c6c1d002375b4506dfb85f: baseline-arm64-clabbe 2026-03-29 17:44:06,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c1d002375b4506dfb85f 2026-03-29 17:44:06,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:44:06,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c1d002375b4506dfb85f HTTP/1.1" 200 None 2026-03-29 17:44:06,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:44:06,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:44:06,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:44:07,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c1d002375b4506dfb85f HTTP/1.1" 200 None 2026-03-29 17:44:07,019 - lava_callback - ERROR - Error processing callback for node 69c6c1d002375b4506dfb85f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46046 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:47:39,649 - lava_callback - INFO - Received callback for node 69c6c2a802375b4506dfb895 INFO: 172.20.0.1:46054 - "POST /node/69c6c2a802375b4506dfb895 HTTP/1.0" 202 Accepted 2026-03-29 17:47:39,655 - lava_callback - INFO - Starting async job submit for node 69c6c2a802375b4506dfb895 2026-03-29 17:47:39,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:47:39,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c2a802375b4506dfb895 HTTP/1.1" 200 None 2026-03-29 17:47:39,671 - lava_callback - DEBUG - Processing node 69c6c2a802375b4506dfb895: baseline-arm64-baylibre 2026-03-29 17:47:39,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c2a802375b4506dfb895 2026-03-29 17:47:39,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:47:39,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c2a802375b4506dfb895 HTTP/1.1" 200 None 2026-03-29 17:47:39,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:47:39,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 17:47:39,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:47:39,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c2a802375b4506dfb895 HTTP/1.1" 200 None 2026-03-29 17:47:39,724 - lava_callback - ERROR - Error processing callback for node 69c6c2a802375b4506dfb895: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54464 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:48:47,224 - lava_callback - INFO - Received callback for node 69c6c2fb02375b4506dfb8a1 INFO: 172.20.0.1:54480 - "POST /node/69c6c2fb02375b4506dfb8a1 HTTP/1.0" 202 Accepted 2026-03-29 17:48:47,230 - lava_callback - INFO - Starting async job submit for node 69c6c2fb02375b4506dfb8a1 2026-03-29 17:48:47,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:48:47,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c2fb02375b4506dfb8a1 HTTP/1.1" 200 None 2026-03-29 17:48:47,247 - lava_callback - DEBUG - Processing node 69c6c2fb02375b4506dfb8a1: baseline-arm64-clabbe 2026-03-29 17:48:47,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c2fb02375b4506dfb8a1 2026-03-29 17:48:47,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:48:47,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c2fb02375b4506dfb8a1 HTTP/1.1" 200 None 2026-03-29 17:48:47,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:48:47,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:48:47,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:48:47,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c2fb02375b4506dfb8a1 HTTP/1.1" 200 None 2026-03-29 17:48:47,323 - lava_callback - ERROR - Error processing callback for node 69c6c2fb02375b4506dfb8a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53932 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:49:07,857 - lava_callback - INFO - Received callback for node 69c9623ff2866272ea51e72e INFO: 172.20.0.1:53946 - "POST /node/69c9623ff2866272ea51e72e HTTP/1.0" 202 Accepted 2026-03-29 17:49:07,864 - lava_callback - INFO - Starting async job submit for node 69c9623ff2866272ea51e72e 2026-03-29 17:49:07,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:49:08,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9623ff2866272ea51e72e HTTP/1.1" 200 None 2026-03-29 17:49:08,112 - lava_callback - DEBUG - Processing node 69c9623ff2866272ea51e72e: baseline-arm 2026-03-29 17:49:08,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:49:08,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:49:08,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c9623ff2866272ea51e72e/log.txt.gz 2026-03-29 17:49:09,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:49:09,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:49:09,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c9623ff2866272ea51e72e/lava-logs.yaml 2026-03-29 17:49:09,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:49:09,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:49:09,729 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c9623ff2866272ea51e72e/lava_callback.json.gz DEBUG: Setup failed for 69c9623ff2866272ea51e72e. Transitting job node result: incomplete -> fail 2026-03-29 17:49:09,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:49:09,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9623ff2866272ea51e72e HTTP/1.1" 200 None 2026-03-29 17:49:09,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:49:09,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 17:49:09,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:49:09,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9623ff2866272ea51e72e HTTP/1.1" 200 None 2026-03-29 17:49:10,070 - lava_callback - INFO - Completed processing callback for node 69c9623ff2866272ea51e72e INFO: 172.20.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57084 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:49:30,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:49:30,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42780 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:50:20,078 - lava_callback - INFO - Received callback for node 69c96650f2866272ea51e825 INFO: 172.20.0.1:55730 - "POST /node/69c96650f2866272ea51e825 HTTP/1.0" 202 Accepted 2026-03-29 17:50:20,085 - lava_callback - INFO - Starting async job submit for node 69c96650f2866272ea51e825 2026-03-29 17:50:20,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:50:20,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96650f2866272ea51e825 HTTP/1.1" 200 None 2026-03-29 17:50:20,101 - lava_callback - DEBUG - Processing node 69c96650f2866272ea51e825: baseline-nfs-arm64 2026-03-29 17:50:20,103 - lava_callback - WARNING - Failed to upload log 2026-03-29 17:50:20,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:50:20,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:50:20,404 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96650f2866272ea51e825/lava-logs.yaml 2026-03-29 17:50:20,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:50:20,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:50:20,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96650f2866272ea51e825/lava_callback.json.gz 2026-03-29 17:50:20,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:50:20,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96650f2866272ea51e825 HTTP/1.1" 200 None 2026-03-29 17:50:20,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:50:20,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:50:20,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:50:20,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96650f2866272ea51e825 HTTP/1.1" 200 None 2026-03-29 17:50:20,615 - lava_callback - INFO - Completed processing callback for node 69c96650f2866272ea51e825 INFO: 172.20.0.1:55742 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:50:30,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:50:30,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55902 - "GET /node/693f3e3924745582e399ef7c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58528 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:52:07,440 - lava_callback - INFO - Received callback for node 69c6c3b302375b4506dfb8db INFO: 172.20.0.1:58542 - "POST /node/69c6c3b302375b4506dfb8db HTTP/1.0" 202 Accepted 2026-03-29 17:52:07,446 - lava_callback - INFO - Starting async job submit for node 69c6c3b302375b4506dfb8db 2026-03-29 17:52:07,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:52:07,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c3b302375b4506dfb8db HTTP/1.1" 200 None 2026-03-29 17:52:07,462 - lava_callback - DEBUG - Processing node 69c6c3b302375b4506dfb8db: baseline-arm64-clabbe 2026-03-29 17:52:07,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c3b302375b4506dfb8db 2026-03-29 17:52:07,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:52:07,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c3b302375b4506dfb8db HTTP/1.1" 200 None 2026-03-29 17:52:07,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:52:07,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:52:07,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:52:07,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c3b302375b4506dfb8db HTTP/1.1" 200 None 2026-03-29 17:52:07,521 - lava_callback - ERROR - Error processing callback for node 69c6c3b302375b4506dfb8db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40088 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:54:27,513 - lava_callback - INFO - Received callback for node 69c6c44902375b4506dfb8ff INFO: 172.20.0.1:40100 - "POST /node/69c6c44902375b4506dfb8ff HTTP/1.0" 202 Accepted 2026-03-29 17:54:27,519 - lava_callback - INFO - Starting async job submit for node 69c6c44902375b4506dfb8ff 2026-03-29 17:54:27,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:54:27,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c44902375b4506dfb8ff HTTP/1.1" 200 None 2026-03-29 17:54:27,535 - lava_callback - DEBUG - Processing node 69c6c44902375b4506dfb8ff: baseline-arm64-clabbe 2026-03-29 17:54:27,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c44902375b4506dfb8ff 2026-03-29 17:54:27,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:54:27,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c44902375b4506dfb8ff HTTP/1.1" 200 None 2026-03-29 17:54:27,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:54:27,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 17:54:27,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:54:27,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c44902375b4506dfb8ff HTTP/1.1" 200 None 2026-03-29 17:54:27,597 - lava_callback - ERROR - Error processing callback for node 69c6c44902375b4506dfb8ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47896 - "GET /node/694acb236c96ed51d316f677%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:56:20,055 - lava_callback - INFO - Received callback for node 69c6c4b702375b4506dfb91b INFO: 172.20.0.1:47212 - "POST /node/69c6c4b702375b4506dfb91b HTTP/1.0" 202 Accepted 2026-03-29 17:56:20,062 - lava_callback - INFO - Starting async job submit for node 69c6c4b702375b4506dfb91b 2026-03-29 17:56:20,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:56:20,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c4b702375b4506dfb91b HTTP/1.1" 200 None 2026-03-29 17:56:20,108 - lava_callback - DEBUG - Processing node 69c6c4b702375b4506dfb91b: baseline-arm64-baylibre 2026-03-29 17:56:20,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c4b702375b4506dfb91b 2026-03-29 17:56:20,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:56:20,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c4b702375b4506dfb91b HTTP/1.1" 200 None 2026-03-29 17:56:20,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:56:20,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 17:56:20,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:56:20,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c4b702375b4506dfb91b HTTP/1.1" 200 None 2026-03-29 17:56:20,172 - lava_callback - ERROR - Error processing callback for node 69c6c4b702375b4506dfb91b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:57:02,806 - lava_callback - INFO - Received callback for node 69c967b7f2866272ea51e863 INFO: 172.20.0.1:47730 - "POST /node/69c967b7f2866272ea51e863 HTTP/1.0" 202 Accepted 2026-03-29 17:57:02,813 - lava_callback - INFO - Starting async job submit for node 69c967b7f2866272ea51e863 2026-03-29 17:57:02,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:02,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c967b7f2866272ea51e863 HTTP/1.1" 200 None 2026-03-29 17:57:02,858 - lava_callback - DEBUG - Processing node 69c967b7f2866272ea51e863: baseline-arm64 2026-03-29 17:57:02,994 - lava_callback - WARNING - Failed to upload log 2026-03-29 17:57:03,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:57:03,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:57:03,281 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c967b7f2866272ea51e863/lava-logs.yaml 2026-03-29 17:57:03,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:57:03,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:57:03,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c967b7f2866272ea51e863/lava_callback.json.gz 2026-03-29 17:57:03,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:03,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c967b7f2866272ea51e863 HTTP/1.1" 200 None 2026-03-29 17:57:03,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:03,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:57:03,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:03,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c967b7f2866272ea51e863 HTTP/1.1" 200 None 2026-03-29 17:57:03,566 - lava_callback - INFO - Completed processing callback for node 69c967b7f2866272ea51e863 INFO: 172.20.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51382 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:57:30,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:30,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51790 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:57:40,142 - lava_callback - INFO - Received callback for node 69c6c4ff02375b4506dfb925 INFO: 172.20.0.1:38658 - "POST /node/69c6c4ff02375b4506dfb925 HTTP/1.0" 202 Accepted 2026-03-29 17:57:40,148 - lava_callback - INFO - Starting async job submit for node 69c6c4ff02375b4506dfb925 2026-03-29 17:57:40,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:40,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c4ff02375b4506dfb925 HTTP/1.1" 200 None 2026-03-29 17:57:40,173 - lava_callback - DEBUG - Processing node 69c6c4ff02375b4506dfb925: baseline-arm64-baylibre 2026-03-29 17:57:40,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c4ff02375b4506dfb925 2026-03-29 17:57:40,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:40,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c4ff02375b4506dfb925 HTTP/1.1" 200 None 2026-03-29 17:57:40,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:40,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 17:57:40,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:57:40,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c4ff02375b4506dfb925 HTTP/1.1" 200 None 2026-03-29 17:57:40,235 - lava_callback - ERROR - Error processing callback for node 69c6c4ff02375b4506dfb925: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34580 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:58:05,645 - lava_callback - INFO - Received callback for node 69c965b0f2866272ea51e808 INFO: 172.20.0.1:40976 - "POST /node/69c965b0f2866272ea51e808 HTTP/1.0" 202 Accepted 2026-03-29 17:58:05,652 - lava_callback - INFO - Starting async job submit for node 69c965b0f2866272ea51e808 2026-03-29 17:58:05,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:05,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c965b0f2866272ea51e808 HTTP/1.1" 200 None 2026-03-29 17:58:05,691 - lava_callback - DEBUG - Processing node 69c965b0f2866272ea51e808: baseline-arm64 2026-03-29 17:58:06,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40988 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:58:06,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:06,561 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c965b0f2866272ea51e808/log.txt.gz 2026-03-29 17:58:06,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:07,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:07,066 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c965b0f2866272ea51e808/lava-logs.yaml 2026-03-29 17:58:07,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:07,238 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:07,240 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c965b0f2866272ea51e808/lava_callback.json.gz 2026-03-29 17:58:07,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:07,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c965b0f2866272ea51e808 HTTP/1.1" 200 None 2026-03-29 17:58:07,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:07,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:58:07,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:07,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c965b0f2866272ea51e808 HTTP/1.1" 200 None 2026-03-29 17:58:07,390 - lava_callback - INFO - Completed processing callback for node 69c965b0f2866272ea51e808 INFO: 172.20.0.1:54218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54236 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:58:16,587 - lava_callback - INFO - Received callback for node 69c963abf2866272ea51e775 INFO: 172.20.0.1:54238 - "POST /node/69c963abf2866272ea51e775 HTTP/1.0" 202 Accepted 2026-03-29 17:58:16,633 - lava_callback - INFO - Starting async job submit for node 69c963abf2866272ea51e775 2026-03-29 17:58:16,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:16,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c963abf2866272ea51e775 HTTP/1.1" 200 None 2026-03-29 17:58:16,674 - lava_callback - DEBUG - Processing node 69c963abf2866272ea51e775: baseline-arm-pengutronix 2026-03-29 17:58:16,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:16,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:16,941 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c963abf2866272ea51e775/log.txt.gz 2026-03-29 17:58:17,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:17,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:17,150 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c963abf2866272ea51e775/lava-logs.yaml 2026-03-29 17:58:17,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:17,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:17,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c963abf2866272ea51e775/lava_callback.json.gz DEBUG: Setup failed for 69c963abf2866272ea51e775. Transitting job node result: incomplete -> fail 2026-03-29 17:58:17,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:17,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c963abf2866272ea51e775 HTTP/1.1" 200 None 2026-03-29 17:58:17,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:17,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 17:58:17,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:17,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c963abf2866272ea51e775 HTTP/1.1" 200 None 2026-03-29 17:58:17,464 - lava_callback - INFO - Completed processing callback for node 69c963abf2866272ea51e775 INFO: 172.20.0.1:45862 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:58:30,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:30,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:58:40,088 - lava_callback - INFO - Received callback for node 69c6c54e02375b4506dfb935 INFO: 172.20.0.1:47550 - "POST /node/69c6c54e02375b4506dfb935 HTTP/1.0" 202 Accepted 2026-03-29 17:58:40,093 - lava_callback - INFO - Starting async job submit for node 69c6c54e02375b4506dfb935 2026-03-29 17:58:40,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:40,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c54e02375b4506dfb935 HTTP/1.1" 200 None 2026-03-29 17:58:40,109 - lava_callback - DEBUG - Processing node 69c6c54e02375b4506dfb935: baseline-arm64-baylibre 2026-03-29 17:58:40,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c54e02375b4506dfb935 2026-03-29 17:58:40,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:40,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c54e02375b4506dfb935 HTTP/1.1" 200 None 2026-03-29 17:58:40,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:40,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 17:58:40,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:40,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c54e02375b4506dfb935 HTTP/1.1" 200 None 2026-03-29 17:58:40,191 - lava_callback - ERROR - Error processing callback for node 69c6c54e02375b4506dfb935: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 17:58:40,698 - lava_callback - INFO - Received callback for node 69c9681bf2866272ea51e870 INFO: 172.20.0.1:47556 - "POST /node/69c9681bf2866272ea51e870 HTTP/1.0" 202 Accepted 2026-03-29 17:58:40,705 - lava_callback - INFO - Starting async job submit for node 69c9681bf2866272ea51e870 2026-03-29 17:58:40,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:40,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9681bf2866272ea51e870 HTTP/1.1" 200 None 2026-03-29 17:58:40,751 - lava_callback - DEBUG - Processing node 69c9681bf2866272ea51e870: baseline-arm64 2026-03-29 17:58:40,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:41,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:41,018 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c9681bf2866272ea51e870/log.txt.gz 2026-03-29 17:58:41,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:41,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:41,243 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c9681bf2866272ea51e870/lava-logs.yaml 2026-03-29 17:58:41,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 17:58:41,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 17:58:41,435 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c9681bf2866272ea51e870/lava_callback.json.gz 2026-03-29 17:58:41,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:41,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9681bf2866272ea51e870 HTTP/1.1" 200 None 2026-03-29 17:58:41,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:41,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 17:58:41,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:58:41,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9681bf2866272ea51e870 HTTP/1.1" 200 None 2026-03-29 17:58:41,626 - lava_callback - INFO - Completed processing callback for node 69c9681bf2866272ea51e870 INFO: 172.20.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48094 - "GET / HTTP/1.0" 200 OK 2026-03-29 17:59:00,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 17:59:00,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET /node/69903591985e4efca431c7ae%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:33290 - "GET /node/699f9a62f53f50eff99c705b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59314 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:00:00,230 - lava_callback - INFO - Received callback for node 69c6c59a02375b4506dfb93e INFO: 172.20.0.1:41452 - "POST /node/69c6c59a02375b4506dfb93e HTTP/1.0" 202 Accepted 2026-03-29 18:00:00,235 - lava_callback - INFO - Starting async job submit for node 69c6c59a02375b4506dfb93e 2026-03-29 18:00:00,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:00:00,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c59a02375b4506dfb93e HTTP/1.1" 200 None 2026-03-29 18:00:00,252 - lava_callback - DEBUG - Processing node 69c6c59a02375b4506dfb93e: baseline-arm64-baylibre 2026-03-29 18:00:00,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c59a02375b4506dfb93e 2026-03-29 18:00:00,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:00:00,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c59a02375b4506dfb93e HTTP/1.1" 200 None 2026-03-29 18:00:00,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:00:00,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 18:00:00,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:00:00,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c59a02375b4506dfb93e HTTP/1.1" 200 None 2026-03-29 18:00:00,321 - lava_callback - ERROR - Error processing callback for node 69c6c59a02375b4506dfb93e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34204 - "GET /node/6940fead460bc8e26511641f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59696 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:04:47,934 - lava_callback - INFO - Received callback for node 69c6c6ab02375b4506dfb977 INFO: 172.20.0.1:59698 - "POST /node/69c6c6ab02375b4506dfb977 HTTP/1.0" 202 Accepted 2026-03-29 18:04:47,939 - lava_callback - INFO - Starting async job submit for node 69c6c6ab02375b4506dfb977 2026-03-29 18:04:47,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:04:47,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c6ab02375b4506dfb977 HTTP/1.1" 200 None 2026-03-29 18:04:47,956 - lava_callback - DEBUG - Processing node 69c6c6ab02375b4506dfb977: baseline-arm64-clabbe 2026-03-29 18:04:47,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c6ab02375b4506dfb977 2026-03-29 18:04:47,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:04:47,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c6ab02375b4506dfb977 HTTP/1.1" 200 None 2026-03-29 18:04:47,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:04:47,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 18:04:47,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:04:48,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c6ab02375b4506dfb977 HTTP/1.1" 200 None 2026-03-29 18:04:48,015 - lava_callback - ERROR - Error processing callback for node 69c6c6ab02375b4506dfb977: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44846 - "GET /node/69b284324bce73bcf02346f0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33640 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:06:00,874 - lava_callback - INFO - Received callback for node 69c969f6f2866272ea51e8c0 INFO: 172.20.0.1:36666 - "POST /node/69c969f6f2866272ea51e8c0 HTTP/1.0" 202 Accepted 2026-03-29 18:06:00,880 - lava_callback - INFO - Starting async job submit for node 69c969f6f2866272ea51e8c0 2026-03-29 18:06:00,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:06:00,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c969f6f2866272ea51e8c0 HTTP/1.1" 200 None 2026-03-29 18:06:00,897 - lava_callback - DEBUG - Processing node 69c969f6f2866272ea51e8c0: baseline-nfs-arm64 2026-03-29 18:06:00,899 - lava_callback - WARNING - Failed to upload log 2026-03-29 18:06:00,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:06:01,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:06:01,040 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c969f6f2866272ea51e8c0/lava-logs.yaml 2026-03-29 18:06:01,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:06:01,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:06:01,172 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c969f6f2866272ea51e8c0/lava_callback.json.gz 2026-03-29 18:06:01,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:06:01,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c969f6f2866272ea51e8c0 HTTP/1.1" 200 None 2026-03-29 18:06:01,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:06:01,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:06:01,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:06:01,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c969f6f2866272ea51e8c0 HTTP/1.1" 200 None 2026-03-29 18:06:01,250 - lava_callback - INFO - Completed processing callback for node 69c969f6f2866272ea51e8c0 INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54458 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:06:30,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:06:30,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:07:30,118 - lava_callback - INFO - Received callback for node 69c96942f2866272ea51e8a7 INFO: 172.20.0.1:54702 - "POST /node/69c96942f2866272ea51e8a7 HTTP/1.0" 202 Accepted 2026-03-29 18:07:30,127 - lava_callback - INFO - Starting async job submit for node 69c96942f2866272ea51e8a7 2026-03-29 18:07:30,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:07:30,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96942f2866272ea51e8a7 HTTP/1.1" 200 None 2026-03-29 18:07:30,195 - lava_callback - DEBUG - Processing node 69c96942f2866272ea51e8a7: baseline-arm64-foundriesio 2026-03-29 18:07:31,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:07:31,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:07:31,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c96942f2866272ea51e8a7/log.txt.gz 2026-03-29 18:07:32,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:07:32,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:07:32,464 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c96942f2866272ea51e8a7/lava-logs.yaml 2026-03-29 18:07:32,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:07:32,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:07:32,850 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c96942f2866272ea51e8a7/lava_callback.json.gz DEBUG: 69c96942f2866272ea51e8a7 Transitting job node result: pass -> incomplete 2026-03-29 18:07:32,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:07:32,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96942f2866272ea51e8a7 HTTP/1.1" 200 None 2026-03-29 18:07:32,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:07:32,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:07:32,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:07:32,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96942f2866272ea51e8a7 HTTP/1.1" 200 None 2026-03-29 18:07:32,972 - lava_callback - INFO - Completed processing callback for node 69c96942f2866272ea51e8a7 INFO: 172.20.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35728 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:08:00,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:00,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53834 - "GET /node/69938047cebe99dd8e06d7a7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53846 - "GET /node/693007f03eb37f3c8fbc17f4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53858 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:08:28,140 - lava_callback - INFO - Received callback for node 69c6c78f02375b4506dfb9a2 INFO: 172.20.0.1:53870 - "POST /node/69c6c78f02375b4506dfb9a2 HTTP/1.0" 202 Accepted 2026-03-29 18:08:28,145 - lava_callback - INFO - Starting async job submit for node 69c6c78f02375b4506dfb9a2 2026-03-29 18:08:28,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:28,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c78f02375b4506dfb9a2 HTTP/1.1" 200 None 2026-03-29 18:08:28,162 - lava_callback - DEBUG - Processing node 69c6c78f02375b4506dfb9a2: baseline-arm64-clabbe 2026-03-29 18:08:28,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c78f02375b4506dfb9a2 2026-03-29 18:08:28,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:28,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c78f02375b4506dfb9a2 HTTP/1.1" 200 None 2026-03-29 18:08:28,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:28,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 18:08:28,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:28,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c78f02375b4506dfb9a2 HTTP/1.1" 200 None 2026-03-29 18:08:28,219 - lava_callback - ERROR - Error processing callback for node 69c6c78f02375b4506dfb9a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54400 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:08:57,081 - lava_callback - INFO - Received callback for node 69c9697cf2866272ea51e8ad INFO: 172.20.0.1:54402 - "POST /node/69c9697cf2866272ea51e8ad HTTP/1.0" 202 Accepted 2026-03-29 18:08:57,089 - lava_callback - INFO - Starting async job submit for node 69c9697cf2866272ea51e8ad 2026-03-29 18:08:57,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:57,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9697cf2866272ea51e8ad HTTP/1.1" 200 None 2026-03-29 18:08:57,145 - lava_callback - DEBUG - Processing node 69c9697cf2866272ea51e8ad: baseline-arm64 2026-03-29 18:08:57,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:08:57,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:08:57,670 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c9697cf2866272ea51e8ad/log.txt.gz 2026-03-29 18:08:57,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:08:58,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:08:58,106 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c9697cf2866272ea51e8ad/lava-logs.yaml 2026-03-29 18:08:58,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:08:58,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:08:58,320 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c9697cf2866272ea51e8ad/lava_callback.json.gz DEBUG: Setup failed for 69c9697cf2866272ea51e8ad. Transitting job node result: incomplete -> fail 2026-03-29 18:08:58,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:58,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9697cf2866272ea51e8ad HTTP/1.1" 200 None 2026-03-29 18:08:58,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:58,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:08:58,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:08:58,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9697cf2866272ea51e8ad HTTP/1.1" 200 None 2026-03-29 18:08:58,530 - lava_callback - INFO - Completed processing callback for node 69c9697cf2866272ea51e8ad 2026-03-29 18:09:00,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:09:00,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:09:22,825 - lava_callback - INFO - Received callback for node 69c96a90f2866272ea51e8d9 INFO: 172.20.0.1:44244 - "POST /node/69c96a90f2866272ea51e8d9 HTTP/1.0" 202 Accepted 2026-03-29 18:09:22,832 - lava_callback - INFO - Starting async job submit for node 69c96a90f2866272ea51e8d9 2026-03-29 18:09:22,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:09:22,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96a90f2866272ea51e8d9 HTTP/1.1" 200 None 2026-03-29 18:09:22,887 - lava_callback - DEBUG - Processing node 69c96a90f2866272ea51e8d9: baseline-arm64 2026-03-29 18:09:23,026 - lava_callback - WARNING - Failed to upload log 2026-03-29 18:09:23,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:09:23,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:09:23,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96a90f2866272ea51e8d9/lava-logs.yaml 2026-03-29 18:09:23,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:09:23,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:09:23,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96a90f2866272ea51e8d9/lava_callback.json.gz 2026-03-29 18:09:23,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:09:23,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96a90f2866272ea51e8d9 HTTP/1.1" 200 None 2026-03-29 18:09:23,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:09:23,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:09:23,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:09:23,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96a90f2866272ea51e8d9 HTTP/1.1" 200 None 2026-03-29 18:09:23,560 - lava_callback - INFO - Completed processing callback for node 69c96a90f2866272ea51e8d9 INFO: 172.20.0.1:44254 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:09:30,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:09:30,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45546 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:11:08,162 - lava_callback - INFO - Received callback for node 69c6c82802375b4506dfb9c7 INFO: 172.20.0.1:45556 - "POST /node/69c6c82802375b4506dfb9c7 HTTP/1.0" 202 Accepted 2026-03-29 18:11:08,168 - lava_callback - INFO - Starting async job submit for node 69c6c82802375b4506dfb9c7 2026-03-29 18:11:08,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:08,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c82802375b4506dfb9c7 HTTP/1.1" 200 None 2026-03-29 18:11:08,184 - lava_callback - DEBUG - Processing node 69c6c82802375b4506dfb9c7: baseline-arm64-clabbe 2026-03-29 18:11:08,184 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c82802375b4506dfb9c7 2026-03-29 18:11:08,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:08,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c82802375b4506dfb9c7 HTTP/1.1" 200 None 2026-03-29 18:11:08,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:08,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 18:11:08,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:08,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c82802375b4506dfb9c7 HTTP/1.1" 200 None 2026-03-29 18:11:08,242 - lava_callback - ERROR - Error processing callback for node 69c6c82802375b4506dfb9c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53046 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:11:58,003 - lava_callback - INFO - Received callback for node 69c968e8f2866272ea51e898 INFO: 172.20.0.1:53062 - "POST /node/69c968e8f2866272ea51e898 HTTP/1.0" 202 Accepted 2026-03-29 18:11:58,010 - lava_callback - INFO - Starting async job submit for node 69c968e8f2866272ea51e898 2026-03-29 18:11:58,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:58,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c968e8f2866272ea51e898 HTTP/1.1" 200 None 2026-03-29 18:11:58,057 - lava_callback - DEBUG - Processing node 69c968e8f2866272ea51e898: baseline-nfs-arm64 2026-03-29 18:11:58,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:11:58,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:11:58,746 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c968e8f2866272ea51e898/log.txt.gz 2026-03-29 18:11:59,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:11:59,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:11:59,395 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c968e8f2866272ea51e898/lava-logs.yaml 2026-03-29 18:11:59,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:11:59,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:11:59,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c968e8f2866272ea51e898/lava_callback.json.gz 2026-03-29 18:11:59,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:59,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c968e8f2866272ea51e898 HTTP/1.1" 200 None 2026-03-29 18:11:59,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:59,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:11:59,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:11:59,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c968e8f2866272ea51e898 HTTP/1.1" 200 None 2026-03-29 18:11:59,716 - lava_callback - INFO - Completed processing callback for node 69c968e8f2866272ea51e898 2026-03-29 18:12:00,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:00,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35814 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:12:08,189 - lava_callback - INFO - Received callback for node 69c6c87502375b4506dfb9d7 INFO: 172.20.0.1:35826 - "POST /node/69c6c87502375b4506dfb9d7 HTTP/1.0" 202 Accepted 2026-03-29 18:12:08,195 - lava_callback - INFO - Starting async job submit for node 69c6c87502375b4506dfb9d7 2026-03-29 18:12:08,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:08,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c87502375b4506dfb9d7 HTTP/1.1" 200 None 2026-03-29 18:12:08,211 - lava_callback - DEBUG - Processing node 69c6c87502375b4506dfb9d7: baseline-arm64-clabbe 2026-03-29 18:12:08,211 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c87502375b4506dfb9d7 2026-03-29 18:12:08,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:08,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c87502375b4506dfb9d7 HTTP/1.1" 200 None 2026-03-29 18:12:08,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:08,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 18:12:08,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:08,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c87502375b4506dfb9d7 HTTP/1.1" 200 None 2026-03-29 18:12:08,289 - lava_callback - ERROR - Error processing callback for node 69c6c87502375b4506dfb9d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60104 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:12:48,512 - lava_callback - INFO - Received callback for node 69c96b63f2866272ea51e8fd INFO: 172.20.0.1:60114 - "POST /node/69c96b63f2866272ea51e8fd HTTP/1.0" 202 Accepted 2026-03-29 18:12:48,519 - lava_callback - INFO - Starting async job submit for node 69c96b63f2866272ea51e8fd 2026-03-29 18:12:48,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:48,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96b63f2866272ea51e8fd HTTP/1.1" 200 None 2026-03-29 18:12:48,587 - lava_callback - DEBUG - Processing node 69c96b63f2866272ea51e8fd: baseline-arm64 2026-03-29 18:12:48,786 - lava_callback - WARNING - Failed to upload log 2026-03-29 18:12:48,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:12:49,134 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:12:49,136 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96b63f2866272ea51e8fd/lava-logs.yaml 2026-03-29 18:12:49,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:12:49,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:12:49,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96b63f2866272ea51e8fd/lava_callback.json.gz 2026-03-29 18:12:49,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:49,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96b63f2866272ea51e8fd HTTP/1.1" 200 None 2026-03-29 18:12:49,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:49,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:12:49,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:12:49,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96b63f2866272ea51e8fd HTTP/1.1" 200 None 2026-03-29 18:12:49,469 - lava_callback - INFO - Completed processing callback for node 69c96b63f2866272ea51e8fd INFO: 172.20.0.1:38902 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:13:00,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:13:00,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44520 - "GET /node/693e5e6d407f32424eb64e5e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44524 - "GET /node/694c6ed92f7ca341fdeb6239%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56800 - "GET /node/69a2ae658020dd22eabee2ca%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET /node/69938431cebe99dd8e06f872%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45754 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:15:48,360 - lava_callback - INFO - Received callback for node 69c6c94a02375b4506dfb9fe INFO: 172.20.0.1:45758 - "POST /node/69c6c94a02375b4506dfb9fe HTTP/1.0" 202 Accepted 2026-03-29 18:15:48,365 - lava_callback - INFO - Starting async job submit for node 69c6c94a02375b4506dfb9fe 2026-03-29 18:15:48,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:15:48,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c94a02375b4506dfb9fe HTTP/1.1" 200 None 2026-03-29 18:15:48,381 - lava_callback - DEBUG - Processing node 69c6c94a02375b4506dfb9fe: baseline-arm64-clabbe 2026-03-29 18:15:48,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6c94a02375b4506dfb9fe 2026-03-29 18:15:48,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:15:48,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6c94a02375b4506dfb9fe HTTP/1.1" 200 None 2026-03-29 18:15:48,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:15:48,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 18:15:48,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:15:48,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6c94a02375b4506dfb9fe HTTP/1.1" 200 None 2026-03-29 18:15:48,450 - lava_callback - ERROR - Error processing callback for node 69c6c94a02375b4506dfb9fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41450 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:16:32,955 - lava_callback - INFO - Received callback for node 69c96bcaf2866272ea51e90e INFO: 172.20.0.1:42298 - "POST /node/69c96bcaf2866272ea51e90e HTTP/1.0" 202 Accepted 2026-03-29 18:16:32,963 - lava_callback - INFO - Starting async job submit for node 69c96bcaf2866272ea51e90e 2026-03-29 18:16:33,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:16:33,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96bcaf2866272ea51e90e HTTP/1.1" 200 None 2026-03-29 18:16:33,020 - lava_callback - DEBUG - Processing node 69c96bcaf2866272ea51e90e: baseline-arm64-foundriesio 2026-03-29 18:16:33,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:16:34,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:16:34,152 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c96bcaf2866272ea51e90e/log.txt.gz 2026-03-29 18:16:34,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:16:34,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:16:34,863 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c96bcaf2866272ea51e90e/lava-logs.yaml 2026-03-29 18:16:34,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:16:35,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:16:35,090 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c96bcaf2866272ea51e90e/lava_callback.json.gz DEBUG: 69c96bcaf2866272ea51e90e Transitting job node result: pass -> incomplete 2026-03-29 18:16:35,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:16:35,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96bcaf2866272ea51e90e HTTP/1.1" 200 None 2026-03-29 18:16:35,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:16:35,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:16:35,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:16:35,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96bcaf2866272ea51e90e HTTP/1.1" 200 None 2026-03-29 18:16:35,194 - lava_callback - INFO - Completed processing callback for node 69c96bcaf2866272ea51e90e INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52306 - "GET /node/69944f555ae459b9df3dceea%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41122 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:17:00,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:17:00,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50878 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:18:49,131 - lava_callback - INFO - Received callback for node 69c9686ff2866272ea51e887 INFO: 172.20.0.1:50882 - "POST /node/69c9686ff2866272ea51e887 HTTP/1.0" 202 Accepted 2026-03-29 18:18:49,139 - lava_callback - INFO - Starting async job submit for node 69c9686ff2866272ea51e887 2026-03-29 18:18:49,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:18:49,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9686ff2866272ea51e887 HTTP/1.1" 200 None 2026-03-29 18:18:49,211 - lava_callback - DEBUG - Processing node 69c9686ff2866272ea51e887: baseline-nfs-arm64 2026-03-29 18:18:50,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:18:50,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:18:50,504 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c9686ff2866272ea51e887/log.txt.gz 2026-03-29 18:18:51,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:18:51,390 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:18:51,392 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c9686ff2866272ea51e887/lava-logs.yaml 2026-03-29 18:18:51,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:18:51,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:18:51,627 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c9686ff2866272ea51e887/lava_callback.json.gz DEBUG: Setup failed for 69c9686ff2866272ea51e887. Transitting job node result: incomplete -> fail 2026-03-29 18:18:51,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:18:51,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9686ff2866272ea51e887 HTTP/1.1" 200 None 2026-03-29 18:18:51,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:18:51,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:18:51,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:18:51,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9686ff2866272ea51e887 HTTP/1.1" 200 None 2026-03-29 18:18:51,854 - lava_callback - INFO - Completed processing callback for node 69c9686ff2866272ea51e887 INFO: 172.20.0.1:54082 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:19:00,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:19:00,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54226 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:19:32,188 - lava_callback - INFO - Received callback for node 69c968dcf2866272ea51e895 INFO: 172.20.0.1:56344 - "POST /node/69c968dcf2866272ea51e895 HTTP/1.0" 202 Accepted 2026-03-29 18:19:32,195 - lava_callback - INFO - Starting async job submit for node 69c968dcf2866272ea51e895 2026-03-29 18:19:32,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:19:32,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c968dcf2866272ea51e895 HTTP/1.1" 200 None 2026-03-29 18:19:32,236 - lava_callback - DEBUG - Processing node 69c968dcf2866272ea51e895: ltp-timers 2026-03-29 18:19:32,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:19:32,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:19:32,716 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c968dcf2866272ea51e895/log.txt.gz 2026-03-29 18:19:32,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:19:33,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:19:33,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c968dcf2866272ea51e895/lava-logs.yaml 2026-03-29 18:19:33,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:19:33,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:19:33,312 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c968dcf2866272ea51e895/lava_callback.json.gz 2026-03-29 18:19:33,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:19:33,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c968dcf2866272ea51e895 HTTP/1.1" 200 None 2026-03-29 18:19:33,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:19:33,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 18:19:33,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:19:33,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c968dcf2866272ea51e895 HTTP/1.1" 200 None 2026-03-29 18:19:33,400 - lava_callback - INFO - Completed processing callback for node 69c968dcf2866272ea51e895 INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40958 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:20:00,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:00,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 18:20:01,879 - lava_callback - INFO - Received callback for node 69c96929f2866272ea51e8a0 INFO: 172.20.0.1:41554 - "POST /node/69c96929f2866272ea51e8a0 HTTP/1.0" 202 Accepted 2026-03-29 18:20:01,886 - lava_callback - INFO - Starting async job submit for node 69c96929f2866272ea51e8a0 2026-03-29 18:20:01,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:01,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96929f2866272ea51e8a0 HTTP/1.1" 200 None 2026-03-29 18:20:01,964 - lava_callback - DEBUG - Processing node 69c96929f2866272ea51e8a0: ltp-smoke 2026-03-29 18:20:02,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:20:02,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:20:02,751 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c96929f2866272ea51e8a0/log.txt.gz 2026-03-29 18:20:03,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:20:03,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:20:03,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c96929f2866272ea51e8a0/lava-logs.yaml 2026-03-29 18:20:03,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:20:03,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:20:03,450 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c96929f2866272ea51e8a0/lava_callback.json.gz 2026-03-29 18:20:03,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:03,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96929f2866272ea51e8a0 HTTP/1.1" 200 None 2026-03-29 18:20:03,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:03,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950aef2866272ea51b8ca HTTP/1.1" 200 None 2026-03-29 18:20:03,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:03,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96929f2866272ea51e8a0 HTTP/1.1" 200 None 2026-03-29 18:20:03,767 - lava_callback - INFO - Completed processing callback for node 69c96929f2866272ea51e8a0 2026-03-29 18:20:05,719 - lava_callback - INFO - Received callback for node 69c96c0af2866272ea51e914 INFO: 172.20.0.1:41568 - "POST /node/69c96c0af2866272ea51e914 HTTP/1.0" 202 Accepted 2026-03-29 18:20:05,726 - lava_callback - INFO - Starting async job submit for node 69c96c0af2866272ea51e914 2026-03-29 18:20:05,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:05,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96c0af2866272ea51e914 HTTP/1.1" 200 None 2026-03-29 18:20:05,777 - lava_callback - DEBUG - Processing node 69c96c0af2866272ea51e914: baseline-arm64 2026-03-29 18:20:06,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:20:06,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:20:06,475 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96c0af2866272ea51e914/log.txt.gz INFO: 172.20.0.1:41582 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:20:06,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:20:06,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:20:06,875 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96c0af2866272ea51e914/lava-logs.yaml 2026-03-29 18:20:06,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:20:07,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:20:07,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c96c0af2866272ea51e914/lava_callback.json.gz DEBUG: Setup failed for 69c96c0af2866272ea51e914. Transitting job node result: incomplete -> fail 2026-03-29 18:20:07,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:07,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96c0af2866272ea51e914 HTTP/1.1" 200 None 2026-03-29 18:20:07,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:07,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:20:07,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:07,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96c0af2866272ea51e914 HTTP/1.1" 200 None 2026-03-29 18:20:07,298 - lava_callback - INFO - Completed processing callback for node 69c96c0af2866272ea51e914 2026-03-29 18:20:08,494 - lava_callback - INFO - Received callback for node 69c6ca5402375b4506dfba32 INFO: 172.20.0.1:41588 - "POST /node/69c6ca5402375b4506dfba32 HTTP/1.0" 202 Accepted 2026-03-29 18:20:08,500 - lava_callback - INFO - Starting async job submit for node 69c6ca5402375b4506dfba32 2026-03-29 18:20:08,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:08,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ca5402375b4506dfba32 HTTP/1.1" 200 None 2026-03-29 18:20:08,517 - lava_callback - DEBUG - Processing node 69c6ca5402375b4506dfba32: baseline-arm64-clabbe 2026-03-29 18:20:08,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ca5402375b4506dfba32 2026-03-29 18:20:08,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:08,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ca5402375b4506dfba32 HTTP/1.1" 200 None 2026-03-29 18:20:08,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:08,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-29 18:20:08,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:08,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ca5402375b4506dfba32 HTTP/1.1" 200 None 2026-03-29 18:20:08,578 - lava_callback - ERROR - Error processing callback for node 69c6ca5402375b4506dfba32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45226 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:20:30,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:20:30,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52278 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:22:36,231 - lava_callback - INFO - Received callback for node 69c969c5f2866272ea51e8b5 INFO: 172.20.0.1:39060 - "POST /node/69c969c5f2866272ea51e8b5 HTTP/1.0" 202 Accepted 2026-03-29 18:22:36,238 - lava_callback - INFO - Starting async job submit for node 69c969c5f2866272ea51e8b5 2026-03-29 18:22:36,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:22:36,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c969c5f2866272ea51e8b5 HTTP/1.1" 200 None 2026-03-29 18:22:36,356 - lava_callback - DEBUG - Processing node 69c969c5f2866272ea51e8b5: baseline-arm INFO: 172.20.0.1:39074 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:22:36,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:22:37,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:22:37,114 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c969c5f2866272ea51e8b5/log.txt.gz 2026-03-29 18:22:37,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:22:37,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:22:37,861 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c969c5f2866272ea51e8b5/lava-logs.yaml 2026-03-29 18:22:37,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:22:38,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:22:38,041 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c969c5f2866272ea51e8b5/lava_callback.json.gz DEBUG: Setup failed for 69c969c5f2866272ea51e8b5. Transitting job node result: incomplete -> fail 2026-03-29 18:22:38,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:22:38,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c969c5f2866272ea51e8b5 HTTP/1.1" 200 None 2026-03-29 18:22:38,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:22:38,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 18:22:38,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:22:38,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c969c5f2866272ea51e8b5 HTTP/1.1" 200 None 2026-03-29 18:22:38,319 - lava_callback - INFO - Completed processing callback for node 69c969c5f2866272ea51e8b5 INFO: 172.20.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60282 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:23:00,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:23:00,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 18:23:01,621 - lava_callback - INFO - Received callback for node 69c6cafd02375b4506dfba53 INFO: 172.20.0.1:43184 - "POST /node/69c6cafd02375b4506dfba53 HTTP/1.0" 202 Accepted 2026-03-29 18:23:01,626 - lava_callback - INFO - Starting async job submit for node 69c6cafd02375b4506dfba53 2026-03-29 18:23:01,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:23:01,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cafd02375b4506dfba53 HTTP/1.1" 200 None 2026-03-29 18:23:01,642 - lava_callback - DEBUG - Processing node 69c6cafd02375b4506dfba53: baseline-arm-baylibre 2026-03-29 18:23:01,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cafd02375b4506dfba53 2026-03-29 18:23:01,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:23:01,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cafd02375b4506dfba53 HTTP/1.1" 200 None 2026-03-29 18:23:01,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:23:01,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:23:01,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:23:01,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cafd02375b4506dfba53 HTTP/1.1" 200 None 2026-03-29 18:23:01,693 - lava_callback - ERROR - Error processing callback for node 69c6cafd02375b4506dfba53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:24:21,759 - lava_callback - INFO - Received callback for node 69c6cb4402375b4506dfba68 INFO: 172.20.0.1:47056 - "POST /node/69c6cb4402375b4506dfba68 HTTP/1.0" 202 Accepted 2026-03-29 18:24:21,765 - lava_callback - INFO - Starting async job submit for node 69c6cb4402375b4506dfba68 2026-03-29 18:24:21,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:24:21,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cb4402375b4506dfba68 HTTP/1.1" 200 None 2026-03-29 18:24:21,796 - lava_callback - DEBUG - Processing node 69c6cb4402375b4506dfba68: baseline-arm-baylibre 2026-03-29 18:24:21,796 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cb4402375b4506dfba68 2026-03-29 18:24:21,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:24:21,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cb4402375b4506dfba68 HTTP/1.1" 200 None 2026-03-29 18:24:21,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:24:21,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:24:21,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:24:21,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cb4402375b4506dfba68 HTTP/1.1" 200 None 2026-03-29 18:24:21,870 - lava_callback - ERROR - Error processing callback for node 69c6cb4402375b4506dfba68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:25:01,594 - lava_callback - INFO - Received callback for node 69c6cb6902375b4506dfba79 INFO: 172.20.0.1:40878 - "POST /node/69c6cb6902375b4506dfba79 HTTP/1.0" 202 Accepted 2026-03-29 18:25:01,600 - lava_callback - INFO - Starting async job submit for node 69c6cb6902375b4506dfba79 2026-03-29 18:25:01,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:01,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cb6902375b4506dfba79 HTTP/1.1" 200 None 2026-03-29 18:25:01,652 - lava_callback - DEBUG - Processing node 69c6cb6902375b4506dfba79: baseline-arm-baylibre 2026-03-29 18:25:01,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cb6902375b4506dfba79 2026-03-29 18:25:01,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:01,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cb6902375b4506dfba79 HTTP/1.1" 200 None 2026-03-29 18:25:01,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:01,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:25:01,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:01,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cb6902375b4506dfba79 HTTP/1.1" 200 None 2026-03-29 18:25:01,730 - lava_callback - ERROR - Error processing callback for node 69c6cb6902375b4506dfba79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:25:21,603 - lava_callback - INFO - Received callback for node 69c6cb8d02375b4506dfba82 INFO: 172.20.0.1:41584 - "POST /node/69c6cb8d02375b4506dfba82 HTTP/1.0" 202 Accepted 2026-03-29 18:25:21,608 - lava_callback - INFO - Starting async job submit for node 69c6cb8d02375b4506dfba82 2026-03-29 18:25:21,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:21,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cb8d02375b4506dfba82 HTTP/1.1" 200 None 2026-03-29 18:25:21,628 - lava_callback - DEBUG - Processing node 69c6cb8d02375b4506dfba82: baseline-arm-baylibre 2026-03-29 18:25:21,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cb8d02375b4506dfba82 2026-03-29 18:25:21,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:21,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cb8d02375b4506dfba82 HTTP/1.1" 200 None 2026-03-29 18:25:21,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:21,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:25:21,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:25:21,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cb8d02375b4506dfba82 HTTP/1.1" 200 None 2026-03-29 18:25:21,687 - lava_callback - ERROR - Error processing callback for node 69c6cb8d02375b4506dfba82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50272 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:26:41,800 - lava_callback - INFO - Received callback for node 69c6cbd502375b4506dfba97 INFO: 172.20.0.1:53988 - "POST /node/69c6cbd502375b4506dfba97 HTTP/1.0" 202 Accepted 2026-03-29 18:26:41,806 - lava_callback - INFO - Starting async job submit for node 69c6cbd502375b4506dfba97 2026-03-29 18:26:41,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:26:41,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cbd502375b4506dfba97 HTTP/1.1" 200 None 2026-03-29 18:26:41,822 - lava_callback - DEBUG - Processing node 69c6cbd502375b4506dfba97: baseline-arm-baylibre 2026-03-29 18:26:41,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cbd502375b4506dfba97 2026-03-29 18:26:41,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:26:41,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cbd502375b4506dfba97 HTTP/1.1" 200 None 2026-03-29 18:26:41,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:26:41,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:26:41,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:26:41,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cbd502375b4506dfba97 HTTP/1.1" 200 None 2026-03-29 18:26:41,875 - lava_callback - ERROR - Error processing callback for node 69c6cbd502375b4506dfba97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45978 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:28:01,833 - lava_callback - INFO - Received callback for node 69c6cc1d02375b4506dfbaaa INFO: 172.20.0.1:43770 - "POST /node/69c6cc1d02375b4506dfbaaa HTTP/1.0" 202 Accepted 2026-03-29 18:28:01,838 - lava_callback - INFO - Starting async job submit for node 69c6cc1d02375b4506dfbaaa 2026-03-29 18:28:01,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:01,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cc1d02375b4506dfbaaa HTTP/1.1" 200 None 2026-03-29 18:28:01,879 - lava_callback - DEBUG - Processing node 69c6cc1d02375b4506dfbaaa: baseline-arm-baylibre 2026-03-29 18:28:01,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cc1d02375b4506dfbaaa 2026-03-29 18:28:01,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:01,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cc1d02375b4506dfbaaa HTTP/1.1" 200 None 2026-03-29 18:28:01,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:01,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:28:01,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:01,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cc1d02375b4506dfbaaa HTTP/1.1" 200 None 2026-03-29 18:28:01,969 - lava_callback - ERROR - Error processing callback for node 69c6cc1d02375b4506dfbaaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38968 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:28:28,862 - lava_callback - INFO - Received callback for node 69c6cc4402375b4506dfbab9 INFO: 172.20.0.1:38974 - "POST /node/69c6cc4402375b4506dfbab9 HTTP/1.0" 202 Accepted 2026-03-29 18:28:28,868 - lava_callback - INFO - Starting async job submit for node 69c6cc4402375b4506dfbab9 2026-03-29 18:28:28,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:28,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cc4402375b4506dfbab9 HTTP/1.1" 200 None 2026-03-29 18:28:28,891 - lava_callback - DEBUG - Processing node 69c6cc4402375b4506dfbab9: baseline-arm-clabbe 2026-03-29 18:28:28,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cc4402375b4506dfbab9 2026-03-29 18:28:28,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:28,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cc4402375b4506dfbab9 HTTP/1.1" 200 None 2026-03-29 18:28:28,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:28,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:28:28,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:28:28,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cc4402375b4506dfbab9 HTTP/1.1" 200 None 2026-03-29 18:28:28,962 - lava_callback - ERROR - Error processing callback for node 69c6cc4402375b4506dfbab9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36932 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:30:29,068 - lava_callback - INFO - Received callback for node 69c6ccb502375b4506dfbad1 INFO: 172.20.0.1:36936 - "POST /node/69c6ccb502375b4506dfbad1 HTTP/1.0" 202 Accepted 2026-03-29 18:30:29,074 - lava_callback - INFO - Starting async job submit for node 69c6ccb502375b4506dfbad1 2026-03-29 18:30:29,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:30:29,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ccb502375b4506dfbad1 HTTP/1.1" 200 None 2026-03-29 18:30:29,091 - lava_callback - DEBUG - Processing node 69c6ccb502375b4506dfbad1: baseline-arm-clabbe 2026-03-29 18:30:29,091 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ccb502375b4506dfbad1 2026-03-29 18:30:29,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:30:29,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ccb502375b4506dfbad1 HTTP/1.1" 200 None 2026-03-29 18:30:29,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:30:29,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:30:29,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:30:29,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ccb502375b4506dfbad1 HTTP/1.1" 200 None 2026-03-29 18:30:29,144 - lava_callback - ERROR - Error processing callback for node 69c6ccb502375b4506dfbad1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42942 - "GET /node/691091d08b15debe9dd50bed%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57346 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:31:08,973 - lava_callback - INFO - Received callback for node 69c6ccdd02375b4506dfbadb INFO: 172.20.0.1:57354 - "POST /node/69c6ccdd02375b4506dfbadb HTTP/1.0" 202 Accepted 2026-03-29 18:31:08,979 - lava_callback - INFO - Starting async job submit for node 69c6ccdd02375b4506dfbadb 2026-03-29 18:31:08,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:08,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ccdd02375b4506dfbadb HTTP/1.1" 200 None 2026-03-29 18:31:08,997 - lava_callback - DEBUG - Processing node 69c6ccdd02375b4506dfbadb: baseline-arm-clabbe 2026-03-29 18:31:08,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ccdd02375b4506dfbadb 2026-03-29 18:31:08,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:09,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ccdd02375b4506dfbadb HTTP/1.1" 200 None 2026-03-29 18:31:09,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:09,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:31:09,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:09,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ccdd02375b4506dfbadb HTTP/1.1" 200 None 2026-03-29 18:31:09,059 - lava_callback - ERROR - Error processing callback for node 69c6ccdd02375b4506dfbadb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39246 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:31:49,074 - lava_callback - INFO - Received callback for node 69c6cd0402375b4506dfbae8 INFO: 172.20.0.1:39260 - "POST /node/69c6cd0402375b4506dfbae8 HTTP/1.0" 202 Accepted 2026-03-29 18:31:49,080 - lava_callback - INFO - Starting async job submit for node 69c6cd0402375b4506dfbae8 2026-03-29 18:31:49,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:49,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cd0402375b4506dfbae8 HTTP/1.1" 200 None 2026-03-29 18:31:49,101 - lava_callback - DEBUG - Processing node 69c6cd0402375b4506dfbae8: baseline-arm-clabbe 2026-03-29 18:31:49,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cd0402375b4506dfbae8 2026-03-29 18:31:49,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:49,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cd0402375b4506dfbae8 HTTP/1.1" 200 None 2026-03-29 18:31:49,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:49,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:31:49,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:31:49,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cd0402375b4506dfbae8 HTTP/1.1" 200 None 2026-03-29 18:31:49,174 - lava_callback - ERROR - Error processing callback for node 69c6cd0402375b4506dfbae8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47526 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:32:29,068 - lava_callback - INFO - Received callback for node 69c6cd2b02375b4506dfbaef INFO: 172.20.0.1:47528 - "POST /node/69c6cd2b02375b4506dfbaef HTTP/1.0" 202 Accepted 2026-03-29 18:32:29,074 - lava_callback - INFO - Starting async job submit for node 69c6cd2b02375b4506dfbaef 2026-03-29 18:32:29,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:32:29,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cd2b02375b4506dfbaef HTTP/1.1" 200 None 2026-03-29 18:32:29,115 - lava_callback - DEBUG - Processing node 69c6cd2b02375b4506dfbaef: baseline-arm-clabbe 2026-03-29 18:32:29,115 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cd2b02375b4506dfbaef 2026-03-29 18:32:29,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:32:29,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cd2b02375b4506dfbaef HTTP/1.1" 200 None 2026-03-29 18:32:29,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:32:29,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-29 18:32:29,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:32:29,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cd2b02375b4506dfbaef HTTP/1.1" 200 None 2026-03-29 18:32:29,183 - lava_callback - ERROR - Error processing callback for node 69c6cd2b02375b4506dfbaef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44868 - "GET /node/69407b71575bc060ce15ba59%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:33:42,301 - lava_callback - INFO - Received callback for node 69c6cd7c02375b4506dfbb0c INFO: 172.20.0.1:53136 - "POST /node/69c6cd7c02375b4506dfbb0c HTTP/1.0" 202 Accepted 2026-03-29 18:33:42,307 - lava_callback - INFO - Starting async job submit for node 69c6cd7c02375b4506dfbb0c 2026-03-29 18:33:42,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:33:42,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cd7c02375b4506dfbb0c HTTP/1.1" 200 None 2026-03-29 18:33:42,324 - lava_callback - DEBUG - Processing node 69c6cd7c02375b4506dfbb0c: baseline-arm64-baylibre 2026-03-29 18:33:42,324 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cd7c02375b4506dfbb0c 2026-03-29 18:33:42,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:33:42,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cd7c02375b4506dfbb0c HTTP/1.1" 200 None 2026-03-29 18:33:42,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:33:42,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:33:42,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:33:42,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cd7c02375b4506dfbb0c HTTP/1.1" 200 None 2026-03-29 18:33:42,390 - lava_callback - ERROR - Error processing callback for node 69c6cd7c02375b4506dfbb0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37336 - "GET /node/68fb3d225818c913a1e19a37%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51598 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:36:22,357 - lava_callback - INFO - Received callback for node 69c6ce1302375b4506dfbb1d INFO: 172.20.0.1:46060 - "POST /node/69c6ce1302375b4506dfbb1d HTTP/1.0" 202 Accepted 2026-03-29 18:36:22,363 - lava_callback - INFO - Starting async job submit for node 69c6ce1302375b4506dfbb1d 2026-03-29 18:36:22,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:22,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ce1302375b4506dfbb1d HTTP/1.1" 200 None 2026-03-29 18:36:22,384 - lava_callback - DEBUG - Processing node 69c6ce1302375b4506dfbb1d: baseline-arm64-baylibre 2026-03-29 18:36:22,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ce1302375b4506dfbb1d 2026-03-29 18:36:22,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:22,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ce1302375b4506dfbb1d HTTP/1.1" 200 None 2026-03-29 18:36:22,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:22,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:36:22,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:22,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ce1302375b4506dfbb1d HTTP/1.1" 200 None 2026-03-29 18:36:22,462 - lava_callback - ERROR - Error processing callback for node 69c6ce1302375b4506dfbb1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42542 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:36:58,000 - lava_callback - INFO - Received callback for node 69c96b44f2866272ea51e8f4 INFO: 172.20.0.1:42546 - "POST /node/69c96b44f2866272ea51e8f4 HTTP/1.0" 202 Accepted 2026-03-29 18:36:58,049 - lava_callback - INFO - Starting async job submit for node 69c96b44f2866272ea51e8f4 2026-03-29 18:36:58,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:58,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96b44f2866272ea51e8f4 HTTP/1.1" 200 None 2026-03-29 18:36:58,089 - lava_callback - DEBUG - Processing node 69c96b44f2866272ea51e8f4: baseline-arm-pengutronix 2026-03-29 18:36:58,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:36:58,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:36:58,380 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c96b44f2866272ea51e8f4/log.txt.gz 2026-03-29 18:36:58,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:36:58,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:36:58,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c96b44f2866272ea51e8f4/lava-logs.yaml 2026-03-29 18:36:58,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:36:58,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:36:58,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c96b44f2866272ea51e8f4/lava_callback.json.gz DEBUG: Setup failed for 69c96b44f2866272ea51e8f4. Transitting job node result: incomplete -> fail 2026-03-29 18:36:58,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:58,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96b44f2866272ea51e8f4 HTTP/1.1" 200 None 2026-03-29 18:36:58,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:58,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 18:36:58,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:36:58,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96b44f2866272ea51e8f4 HTTP/1.1" 200 None 2026-03-29 18:36:58,955 - lava_callback - INFO - Completed processing callback for node 69c96b44f2866272ea51e8f4 2026-03-29 18:37:00,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:00,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33954 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:37:22,254 - lava_callback - INFO - Received callback for node 69c6ce5e02375b4506dfbb2d INFO: 172.20.0.1:39502 - "POST /node/69c6ce5e02375b4506dfbb2d HTTP/1.0" 202 Accepted 2026-03-29 18:37:22,260 - lava_callback - INFO - Starting async job submit for node 69c6ce5e02375b4506dfbb2d 2026-03-29 18:37:22,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:22,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ce5e02375b4506dfbb2d HTTP/1.1" 200 None 2026-03-29 18:37:22,277 - lava_callback - DEBUG - Processing node 69c6ce5e02375b4506dfbb2d: baseline-arm64-baylibre 2026-03-29 18:37:22,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ce5e02375b4506dfbb2d 2026-03-29 18:37:22,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:22,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ce5e02375b4506dfbb2d HTTP/1.1" 200 None 2026-03-29 18:37:22,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:22,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:37:22,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:22,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ce5e02375b4506dfbb2d HTTP/1.1" 200 None 2026-03-29 18:37:22,345 - lava_callback - ERROR - Error processing callback for node 69c6ce5e02375b4506dfbb2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57962 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:37:42,569 - lava_callback - INFO - Received callback for node 69c96d5ff2866272ea51e948 INFO: 172.20.0.1:48084 - "POST /node/69c96d5ff2866272ea51e948 HTTP/1.0" 202 Accepted 2026-03-29 18:37:42,576 - lava_callback - INFO - Starting async job submit for node 69c96d5ff2866272ea51e948 2026-03-29 18:37:42,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:42,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96d5ff2866272ea51e948 HTTP/1.1" 200 None 2026-03-29 18:37:42,649 - lava_callback - DEBUG - Processing node 69c96d5ff2866272ea51e948: ltp-smoke 2026-03-29 18:37:43,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:37:43,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:37:43,277 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c96d5ff2866272ea51e948/log.txt.gz 2026-03-29 18:37:43,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:37:43,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:37:43,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c96d5ff2866272ea51e948/lava-logs.yaml 2026-03-29 18:37:43,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:37:43,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:37:43,960 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c96d5ff2866272ea51e948/lava_callback.json.gz 2026-03-29 18:37:44,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:44,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96d5ff2866272ea51e948 HTTP/1.1" 200 None 2026-03-29 18:37:44,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:44,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950aef2866272ea51b8ca HTTP/1.1" 200 None 2026-03-29 18:37:44,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:37:44,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96d5ff2866272ea51e948 HTTP/1.1" 200 None 2026-03-29 18:37:44,281 - lava_callback - INFO - Completed processing callback for node 69c96d5ff2866272ea51e948 INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42156 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:38:00,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:38:00,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55670 - "GET /node/6996d2039b00d91f7d3e0866%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56080 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:38:42,522 - lava_callback - INFO - Received callback for node 69c6ceac02375b4506dfbb3e INFO: 172.20.0.1:44558 - "POST /node/69c6ceac02375b4506dfbb3e HTTP/1.0" 202 Accepted 2026-03-29 18:38:42,528 - lava_callback - INFO - Starting async job submit for node 69c6ceac02375b4506dfbb3e 2026-03-29 18:38:42,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:38:42,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ceac02375b4506dfbb3e HTTP/1.1" 200 None 2026-03-29 18:38:42,543 - lava_callback - DEBUG - Processing node 69c6ceac02375b4506dfbb3e: baseline-arm64-baylibre 2026-03-29 18:38:42,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ceac02375b4506dfbb3e 2026-03-29 18:38:42,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:38:42,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ceac02375b4506dfbb3e HTTP/1.1" 200 None 2026-03-29 18:38:42,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:38:42,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:38:42,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:38:42,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ceac02375b4506dfbb3e HTTP/1.1" 200 None 2026-03-29 18:38:42,600 - lava_callback - ERROR - Error processing callback for node 69c6ceac02375b4506dfbb3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52300 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:39:11,641 - lava_callback - INFO - Received callback for node 69c96d14f2866272ea51e931 INFO: 172.20.0.1:48488 - "POST /node/69c96d14f2866272ea51e931 HTTP/1.0" 202 Accepted 2026-03-29 18:39:11,649 - lava_callback - INFO - Starting async job submit for node 69c96d14f2866272ea51e931 2026-03-29 18:39:11,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:39:11,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96d14f2866272ea51e931 HTTP/1.1" 200 None 2026-03-29 18:39:11,709 - lava_callback - DEBUG - Processing node 69c96d14f2866272ea51e931: baseline-nfs-arm64 INFO: 172.20.0.1:48504 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:39:12,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:39:13,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:39:13,078 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96d14f2866272ea51e931/log.txt.gz 2026-03-29 18:39:13,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:39:13,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:39:13,981 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96d14f2866272ea51e931/lava-logs.yaml 2026-03-29 18:39:14,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:39:14,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:39:14,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c96d14f2866272ea51e931/lava_callback.json.gz DEBUG: Setup failed for 69c96d14f2866272ea51e931. Transitting job node result: incomplete -> fail 2026-03-29 18:39:14,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:39:14,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96d14f2866272ea51e931 HTTP/1.1" 200 None 2026-03-29 18:39:14,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:39:14,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 18:39:14,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:39:14,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96d14f2866272ea51e931 HTTP/1.1" 200 None 2026-03-29 18:39:14,453 - lava_callback - INFO - Completed processing callback for node 69c96d14f2866272ea51e931 INFO: 172.20.0.1:48510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50884 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:39:30,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:39:30,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:40:02,433 - lava_callback - INFO - Received callback for node 69c6cef702375b4506dfbb4b INFO: 172.20.0.1:58540 - "POST /node/69c6cef702375b4506dfbb4b HTTP/1.0" 202 Accepted 2026-03-29 18:40:02,439 - lava_callback - INFO - Starting async job submit for node 69c6cef702375b4506dfbb4b 2026-03-29 18:40:02,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:02,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cef702375b4506dfbb4b HTTP/1.1" 200 None 2026-03-29 18:40:02,455 - lava_callback - DEBUG - Processing node 69c6cef702375b4506dfbb4b: baseline-arm64-baylibre 2026-03-29 18:40:02,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cef702375b4506dfbb4b 2026-03-29 18:40:02,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:02,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cef702375b4506dfbb4b HTTP/1.1" 200 None 2026-03-29 18:40:02,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:02,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:40:02,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:02,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cef702375b4506dfbb4b HTTP/1.1" 200 None 2026-03-29 18:40:02,520 - lava_callback - ERROR - Error processing callback for node 69c6cef702375b4506dfbb4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:40:37,287 - lava_callback - INFO - Received callback for node 69c96deff2866272ea51e957 INFO: 172.20.0.1:55632 - "POST /node/69c96deff2866272ea51e957 HTTP/1.0" 202 Accepted 2026-03-29 18:40:37,294 - lava_callback - INFO - Starting async job submit for node 69c96deff2866272ea51e957 2026-03-29 18:40:37,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:37,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96deff2866272ea51e957 HTTP/1.1" 200 None 2026-03-29 18:40:37,388 - lava_callback - DEBUG - Processing node 69c96deff2866272ea51e957: baseline-arm 2026-03-29 18:40:37,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:40:38,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:40:38,116 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c96deff2866272ea51e957/log.txt.gz 2026-03-29 18:40:38,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:40:38,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:40:38,732 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c96deff2866272ea51e957/lava-logs.yaml 2026-03-29 18:40:38,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:40:38,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:40:38,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c96deff2866272ea51e957/lava_callback.json.gz DEBUG: Setup failed for 69c96deff2866272ea51e957. Transitting job node result: incomplete -> fail 2026-03-29 18:40:39,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:39,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96deff2866272ea51e957 HTTP/1.1" 200 None 2026-03-29 18:40:39,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:39,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 18:40:39,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:40:39,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96deff2866272ea51e957 HTTP/1.1" 200 None 2026-03-29 18:40:39,177 - lava_callback - INFO - Completed processing callback for node 69c96deff2866272ea51e957 INFO: 172.20.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35106 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:41:00,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:41:00,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36330 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:41:22,685 - lava_callback - INFO - Received callback for node 69c6cf4502375b4506dfbb5f INFO: 172.20.0.1:37016 - "POST /node/69c6cf4502375b4506dfbb5f HTTP/1.0" 202 Accepted 2026-03-29 18:41:22,690 - lava_callback - INFO - Starting async job submit for node 69c6cf4502375b4506dfbb5f 2026-03-29 18:41:22,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:41:22,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cf4502375b4506dfbb5f HTTP/1.1" 200 None 2026-03-29 18:41:22,706 - lava_callback - DEBUG - Processing node 69c6cf4502375b4506dfbb5f: baseline-arm64-baylibre 2026-03-29 18:41:22,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cf4502375b4506dfbb5f 2026-03-29 18:41:22,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:41:22,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cf4502375b4506dfbb5f HTTP/1.1" 200 None 2026-03-29 18:41:22,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:41:22,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:41:22,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:41:22,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cf4502375b4506dfbb5f HTTP/1.1" 200 None 2026-03-29 18:41:22,762 - lava_callback - ERROR - Error processing callback for node 69c6cf4502375b4506dfbb5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55260 - "GET /node/699381cccebe99dd8e06e498%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55266 - "GET /node/69966013818a4357e301117d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55712 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:43:49,591 - lava_callback - INFO - Received callback for node 69c6cfe202375b4506dfbb81 INFO: 172.20.0.1:55726 - "POST /node/69c6cfe202375b4506dfbb81 HTTP/1.0" 202 Accepted 2026-03-29 18:43:49,597 - lava_callback - INFO - Starting async job submit for node 69c6cfe202375b4506dfbb81 2026-03-29 18:43:49,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:43:49,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cfe202375b4506dfbb81 HTTP/1.1" 200 None 2026-03-29 18:43:49,613 - lava_callback - DEBUG - Processing node 69c6cfe202375b4506dfbb81: baseline-arm64-clabbe 2026-03-29 18:43:49,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6cfe202375b4506dfbb81 2026-03-29 18:43:49,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:43:49,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6cfe202375b4506dfbb81 HTTP/1.1" 200 None 2026-03-29 18:43:49,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:43:49,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:43:49,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:43:49,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6cfe202375b4506dfbb81 HTTP/1.1" 200 None 2026-03-29 18:43:49,668 - lava_callback - ERROR - Error processing callback for node 69c6cfe202375b4506dfbb81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44646 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:45:09,611 - lava_callback - INFO - Received callback for node 69c6d02e02375b4506dfbb95 INFO: 172.20.0.1:44658 - "POST /node/69c6d02e02375b4506dfbb95 HTTP/1.0" 202 Accepted 2026-03-29 18:45:09,626 - lava_callback - INFO - Starting async job submit for node 69c6d02e02375b4506dfbb95 2026-03-29 18:45:09,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:45:09,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d02e02375b4506dfbb95 HTTP/1.1" 200 None 2026-03-29 18:45:09,654 - lava_callback - DEBUG - Processing node 69c6d02e02375b4506dfbb95: baseline-arm64-clabbe 2026-03-29 18:45:09,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d02e02375b4506dfbb95 2026-03-29 18:45:09,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:45:09,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d02e02375b4506dfbb95 HTTP/1.1" 200 None 2026-03-29 18:45:09,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:45:09,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:45:09,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:45:09,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d02e02375b4506dfbb95 HTTP/1.1" 200 None 2026-03-29 18:45:09,775 - lava_callback - ERROR - Error processing callback for node 69c6d02e02375b4506dfbb95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51408 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:46:29,717 - lava_callback - INFO - Received callback for node 69c6d07b02375b4506dfbba0 INFO: 172.20.0.1:45972 - "POST /node/69c6d07b02375b4506dfbba0 HTTP/1.0" 202 Accepted 2026-03-29 18:46:29,723 - lava_callback - INFO - Starting async job submit for node 69c6d07b02375b4506dfbba0 2026-03-29 18:46:29,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:46:29,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d07b02375b4506dfbba0 HTTP/1.1" 200 None 2026-03-29 18:46:29,738 - lava_callback - DEBUG - Processing node 69c6d07b02375b4506dfbba0: baseline-arm64-clabbe 2026-03-29 18:46:29,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d07b02375b4506dfbba0 2026-03-29 18:46:29,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:46:29,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d07b02375b4506dfbba0 HTTP/1.1" 200 None 2026-03-29 18:46:29,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:46:29,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:46:29,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:46:29,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d07b02375b4506dfbba0 HTTP/1.1" 200 None 2026-03-29 18:46:29,790 - lava_callback - ERROR - Error processing callback for node 69c6d07b02375b4506dfbba0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46354 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:49:09,874 - lava_callback - INFO - Received callback for node 69c6d11402375b4506dfbbc1 INFO: 172.20.0.1:52968 - "POST /node/69c6d11402375b4506dfbbc1 HTTP/1.0" 202 Accepted 2026-03-29 18:49:09,880 - lava_callback - INFO - Starting async job submit for node 69c6d11402375b4506dfbbc1 2026-03-29 18:49:09,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:49:09,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d11402375b4506dfbbc1 HTTP/1.1" 200 None 2026-03-29 18:49:09,914 - lava_callback - DEBUG - Processing node 69c6d11402375b4506dfbbc1: baseline-arm64-clabbe 2026-03-29 18:49:09,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d11402375b4506dfbbc1 2026-03-29 18:49:09,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:49:09,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d11402375b4506dfbbc1 HTTP/1.1" 200 None 2026-03-29 18:49:09,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:49:09,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:49:09,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:49:09,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d11402375b4506dfbbc1 HTTP/1.1" 200 None 2026-03-29 18:49:09,980 - lava_callback - ERROR - Error processing callback for node 69c6d11402375b4506dfbbc1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48862 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:50:29,867 - lava_callback - INFO - Received callback for node 69c6d16202375b4506dfbbd6 INFO: 172.20.0.1:33262 - "POST /node/69c6d16202375b4506dfbbd6 HTTP/1.0" 202 Accepted 2026-03-29 18:50:29,873 - lava_callback - INFO - Starting async job submit for node 69c6d16202375b4506dfbbd6 2026-03-29 18:50:29,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:50:29,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d16202375b4506dfbbd6 HTTP/1.1" 200 None 2026-03-29 18:50:29,920 - lava_callback - DEBUG - Processing node 69c6d16202375b4506dfbbd6: baseline-arm64-clabbe 2026-03-29 18:50:29,921 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d16202375b4506dfbbd6 2026-03-29 18:50:29,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:50:29,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d16202375b4506dfbbd6 HTTP/1.1" 200 None 2026-03-29 18:50:29,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:50:29,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:50:29,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:50:29,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d16202375b4506dfbbd6 HTTP/1.1" 200 None 2026-03-29 18:50:29,994 - lava_callback - ERROR - Error processing callback for node 69c6d16202375b4506dfbbd6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60118 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:51:49,930 - lava_callback - INFO - Received callback for node 69c6d1b002375b4506dfbbf8 INFO: 172.20.0.1:34282 - "POST /node/69c6d1b002375b4506dfbbf8 HTTP/1.0" 202 Accepted 2026-03-29 18:51:49,936 - lava_callback - INFO - Starting async job submit for node 69c6d1b002375b4506dfbbf8 2026-03-29 18:51:49,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:51:49,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d1b002375b4506dfbbf8 HTTP/1.1" 200 None 2026-03-29 18:51:49,958 - lava_callback - DEBUG - Processing node 69c6d1b002375b4506dfbbf8: baseline-arm64-clabbe 2026-03-29 18:51:49,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d1b002375b4506dfbbf8 2026-03-29 18:51:49,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:51:49,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d1b002375b4506dfbbf8 HTTP/1.1" 200 None 2026-03-29 18:51:49,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:51:49,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-29 18:51:49,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:51:50,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d1b002375b4506dfbbf8 HTTP/1.1" 200 None 2026-03-29 18:51:50,014 - lava_callback - ERROR - Error processing callback for node 69c6d1b002375b4506dfbbf8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54136 - "GET /node/69937332cebe99dd8e069df8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33950 - "GET /node/698771f9e31b58f834f1a7d4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET /node/698a89ebd9a1cfa777a23f2f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46582 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:55:24,035 - lava_callback - INFO - Received callback for node 69c6d29802375b4506dfbc2b INFO: 172.20.0.1:47258 - "POST /node/69c6d29802375b4506dfbc2b HTTP/1.0" 202 Accepted 2026-03-29 18:55:24,040 - lava_callback - INFO - Starting async job submit for node 69c6d29802375b4506dfbc2b 2026-03-29 18:55:24,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:55:24,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d29802375b4506dfbc2b HTTP/1.1" 200 None 2026-03-29 18:55:24,057 - lava_callback - DEBUG - Processing node 69c6d29802375b4506dfbc2b: baseline-arm-baylibre 2026-03-29 18:55:24,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d29802375b4506dfbc2b 2026-03-29 18:55:24,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:55:24,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d29802375b4506dfbc2b HTTP/1.1" 200 None 2026-03-29 18:55:24,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:55:24,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 18:55:24,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:55:24,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d29802375b4506dfbc2b HTTP/1.1" 200 None 2026-03-29 18:55:24,126 - lava_callback - ERROR - Error processing callback for node 69c6d29802375b4506dfbc2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33418 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:56:03,266 - lava_callback - INFO - Received callback for node 69c6d2bd02375b4506dfbc37 INFO: 172.20.0.1:33796 - "POST /node/69c6d2bd02375b4506dfbc37 HTTP/1.0" 202 Accepted 2026-03-29 18:56:03,271 - lava_callback - INFO - Starting async job submit for node 69c6d2bd02375b4506dfbc37 2026-03-29 18:56:03,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:03,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d2bd02375b4506dfbc37 HTTP/1.1" 200 None 2026-03-29 18:56:03,287 - lava_callback - DEBUG - Processing node 69c6d2bd02375b4506dfbc37: baseline-arm-baylibre 2026-03-29 18:56:03,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d2bd02375b4506dfbc37 2026-03-29 18:56:03,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:03,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d2bd02375b4506dfbc37 HTTP/1.1" 200 None 2026-03-29 18:56:03,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:03,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 18:56:03,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:03,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d2bd02375b4506dfbc37 HTTP/1.1" 200 None 2026-03-29 18:56:03,348 - lava_callback - ERROR - Error processing callback for node 69c6d2bd02375b4506dfbc37: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55704 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:56:43,365 - lava_callback - INFO - Received callback for node 69c6d2e302375b4506dfbc42 INFO: 172.20.0.1:34058 - "POST /node/69c6d2e302375b4506dfbc42 HTTP/1.0" 202 Accepted 2026-03-29 18:56:43,370 - lava_callback - INFO - Starting async job submit for node 69c6d2e302375b4506dfbc42 2026-03-29 18:56:43,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:43,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d2e302375b4506dfbc42 HTTP/1.1" 200 None 2026-03-29 18:56:43,402 - lava_callback - DEBUG - Processing node 69c6d2e302375b4506dfbc42: baseline-arm-baylibre 2026-03-29 18:56:43,403 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d2e302375b4506dfbc42 2026-03-29 18:56:43,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:43,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d2e302375b4506dfbc42 HTTP/1.1" 200 None 2026-03-29 18:56:43,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:43,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 18:56:43,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:56:43,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d2e302375b4506dfbc42 HTTP/1.1" 200 None 2026-03-29 18:56:43,480 - lava_callback - ERROR - Error processing callback for node 69c6d2e302375b4506dfbc42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42776 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:57:18,540 - lava_callback - INFO - Received callback for node 69c97182f2866272ea51e99c INFO: 172.20.0.1:42780 - "POST /node/69c97182f2866272ea51e99c HTTP/1.0" 202 Accepted 2026-03-29 18:57:18,590 - lava_callback - INFO - Starting async job submit for node 69c97182f2866272ea51e99c 2026-03-29 18:57:18,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:18,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97182f2866272ea51e99c HTTP/1.1" 200 None 2026-03-29 18:57:18,653 - lava_callback - DEBUG - Processing node 69c97182f2866272ea51e99c: baseline-arm-pengutronix 2026-03-29 18:57:18,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:57:18,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:57:18,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c97182f2866272ea51e99c/log.txt.gz 2026-03-29 18:57:18,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:57:19,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:57:19,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c97182f2866272ea51e99c/lava-logs.yaml 2026-03-29 18:57:19,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 18:57:19,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 18:57:19,420 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c97182f2866272ea51e99c/lava_callback.json.gz DEBUG: Setup failed for 69c97182f2866272ea51e99c. Transitting job node result: incomplete -> fail 2026-03-29 18:57:19,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:19,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97182f2866272ea51e99c HTTP/1.1" 200 None 2026-03-29 18:57:19,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:19,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-29 18:57:19,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:19,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c97182f2866272ea51e99c HTTP/1.1" 200 None 2026-03-29 18:57:19,572 - lava_callback - INFO - Completed processing callback for node 69c97182f2866272ea51e99c 2026-03-29 18:57:23,382 - lava_callback - INFO - Received callback for node 69c6d30a02375b4506dfbc4d INFO: 172.20.0.1:35330 - "POST /node/69c6d30a02375b4506dfbc4d HTTP/1.0" 202 Accepted 2026-03-29 18:57:23,388 - lava_callback - INFO - Starting async job submit for node 69c6d30a02375b4506dfbc4d 2026-03-29 18:57:23,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:23,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d30a02375b4506dfbc4d HTTP/1.1" 200 None 2026-03-29 18:57:23,403 - lava_callback - DEBUG - Processing node 69c6d30a02375b4506dfbc4d: baseline-arm-baylibre 2026-03-29 18:57:23,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d30a02375b4506dfbc4d 2026-03-29 18:57:23,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:23,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d30a02375b4506dfbc4d HTTP/1.1" 200 None 2026-03-29 18:57:23,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:23,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 18:57:23,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:23,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d30a02375b4506dfbc4d HTTP/1.1" 200 None 2026-03-29 18:57:23,466 - lava_callback - ERROR - Error processing callback for node 69c6d30a02375b4506dfbc4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:57:30,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:57:30,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43020 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:58:03,434 - lava_callback - INFO - Received callback for node 69c6d33002375b4506dfbc5a INFO: 172.20.0.1:57196 - "POST /node/69c6d33002375b4506dfbc5a HTTP/1.0" 202 Accepted 2026-03-29 18:58:03,439 - lava_callback - INFO - Starting async job submit for node 69c6d33002375b4506dfbc5a 2026-03-29 18:58:03,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:58:03,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d33002375b4506dfbc5a HTTP/1.1" 200 None 2026-03-29 18:58:03,455 - lava_callback - DEBUG - Processing node 69c6d33002375b4506dfbc5a: baseline-arm-baylibre 2026-03-29 18:58:03,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d33002375b4506dfbc5a 2026-03-29 18:58:03,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:58:03,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d33002375b4506dfbc5a HTTP/1.1" 200 None 2026-03-29 18:58:03,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:58:03,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 18:58:03,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:58:03,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d33002375b4506dfbc5a HTTP/1.1" 200 None 2026-03-29 18:58:03,509 - lava_callback - ERROR - Error processing callback for node 69c6d33002375b4506dfbc5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57816 - "GET / HTTP/1.0" 200 OK 2026-03-29 18:59:50,301 - lava_callback - INFO - Received callback for node 69c6d3a002375b4506dfbc80 INFO: 172.20.0.1:57586 - "POST /node/69c6d3a002375b4506dfbc80 HTTP/1.0" 202 Accepted 2026-03-29 18:59:50,307 - lava_callback - INFO - Starting async job submit for node 69c6d3a002375b4506dfbc80 2026-03-29 18:59:50,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:59:50,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d3a002375b4506dfbc80 HTTP/1.1" 200 None 2026-03-29 18:59:50,339 - lava_callback - DEBUG - Processing node 69c6d3a002375b4506dfbc80: baseline-arm-clabbe 2026-03-29 18:59:50,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d3a002375b4506dfbc80 2026-03-29 18:59:50,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:59:50,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d3a002375b4506dfbc80 HTTP/1.1" 200 None 2026-03-29 18:59:50,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:59:50,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 18:59:50,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 18:59:50,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d3a002375b4506dfbc80 HTTP/1.1" 200 None 2026-03-29 18:59:50,395 - lava_callback - ERROR - Error processing callback for node 69c6d3a002375b4506dfbc80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55616 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:00:30,276 - lava_callback - INFO - Received callback for node 69c6d3c702375b4506dfbc89 INFO: 172.20.0.1:34716 - "POST /node/69c6d3c702375b4506dfbc89 HTTP/1.0" 202 Accepted 2026-03-29 19:00:30,282 - lava_callback - INFO - Starting async job submit for node 69c6d3c702375b4506dfbc89 2026-03-29 19:00:30,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:00:30,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d3c702375b4506dfbc89 HTTP/1.1" 200 None 2026-03-29 19:00:30,299 - lava_callback - DEBUG - Processing node 69c6d3c702375b4506dfbc89: baseline-arm-clabbe 2026-03-29 19:00:30,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d3c702375b4506dfbc89 2026-03-29 19:00:30,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:00:30,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d3c702375b4506dfbc89 HTTP/1.1" 200 None 2026-03-29 19:00:30,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:00:30,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 19:00:30,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:00:30,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d3c702375b4506dfbc89 HTTP/1.1" 200 None 2026-03-29 19:00:30,351 - lava_callback - ERROR - Error processing callback for node 69c6d3c702375b4506dfbc89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:01:10,337 - lava_callback - INFO - Received callback for node 69c6d3ed02375b4506dfbc94 INFO: 172.20.0.1:34848 - "POST /node/69c6d3ed02375b4506dfbc94 HTTP/1.0" 202 Accepted 2026-03-29 19:01:10,343 - lava_callback - INFO - Starting async job submit for node 69c6d3ed02375b4506dfbc94 2026-03-29 19:01:10,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:01:10,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d3ed02375b4506dfbc94 HTTP/1.1" 200 None 2026-03-29 19:01:10,359 - lava_callback - DEBUG - Processing node 69c6d3ed02375b4506dfbc94: baseline-arm-clabbe 2026-03-29 19:01:10,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d3ed02375b4506dfbc94 2026-03-29 19:01:10,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:01:10,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d3ed02375b4506dfbc94 HTTP/1.1" 200 None 2026-03-29 19:01:10,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:01:10,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 19:01:10,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:01:10,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d3ed02375b4506dfbc94 HTTP/1.1" 200 None 2026-03-29 19:01:10,414 - lava_callback - ERROR - Error processing callback for node 69c6d3ed02375b4506dfbc94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:03:10,388 - lava_callback - INFO - Received callback for node 69c6d46402375b4506dfbcb0 INFO: 172.20.0.1:43760 - "POST /node/69c6d46402375b4506dfbcb0 HTTP/1.0" 202 Accepted 2026-03-29 19:03:10,393 - lava_callback - INFO - Starting async job submit for node 69c6d46402375b4506dfbcb0 2026-03-29 19:03:10,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:03:10,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d46402375b4506dfbcb0 HTTP/1.1" 200 None 2026-03-29 19:03:10,430 - lava_callback - DEBUG - Processing node 69c6d46402375b4506dfbcb0: baseline-arm-clabbe 2026-03-29 19:03:10,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d46402375b4506dfbcb0 2026-03-29 19:03:10,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:03:10,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d46402375b4506dfbcb0 HTTP/1.1" 200 None 2026-03-29 19:03:10,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:03:10,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 19:03:10,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:03:10,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d46402375b4506dfbcb0 HTTP/1.1" 200 None 2026-03-29 19:03:10,487 - lava_callback - ERROR - Error processing callback for node 69c6d46402375b4506dfbcb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38144 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:04:30,604 - lava_callback - INFO - Received callback for node 69c6d4af02375b4506dfbcc5 INFO: 172.20.0.1:49196 - "POST /node/69c6d4af02375b4506dfbcc5 HTTP/1.0" 202 Accepted 2026-03-29 19:04:30,610 - lava_callback - INFO - Starting async job submit for node 69c6d4af02375b4506dfbcc5 2026-03-29 19:04:30,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:04:30,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d4af02375b4506dfbcc5 HTTP/1.1" 200 None 2026-03-29 19:04:30,635 - lava_callback - DEBUG - Processing node 69c6d4af02375b4506dfbcc5: baseline-arm-clabbe 2026-03-29 19:04:30,635 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d4af02375b4506dfbcc5 2026-03-29 19:04:30,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:04:30,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d4af02375b4506dfbcc5 HTTP/1.1" 200 None 2026-03-29 19:04:30,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:04:30,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 19:04:30,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:04:30,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d4af02375b4506dfbcc5 HTTP/1.1" 200 None 2026-03-29 19:04:30,746 - lava_callback - ERROR - Error processing callback for node 69c6d4af02375b4506dfbcc5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET /node/6948066b9214b49387165601%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48400 - "GET /node/698ef6def0e3ffbba67d3260%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47808 - "GET /node/69892e1d85773790cd34b626%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47822 - "GET /node/6989360f85773790cd34ca43%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59568 - "GET /node/6985c59f71989fb141498601%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55500 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:13:44,342 - lava_callback - INFO - Received callback for node 69c6d6d602375b4506dfbd24 INFO: 172.20.0.1:60016 - "POST /node/69c6d6d602375b4506dfbd24 HTTP/1.0" 202 Accepted 2026-03-29 19:13:44,348 - lava_callback - INFO - Starting async job submit for node 69c6d6d602375b4506dfbd24 2026-03-29 19:13:44,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:13:44,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d6d602375b4506dfbd24 HTTP/1.1" 200 None 2026-03-29 19:13:44,377 - lava_callback - DEBUG - Processing node 69c6d6d602375b4506dfbd24: baseline-arm64-baylibre 2026-03-29 19:13:44,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d6d602375b4506dfbd24 2026-03-29 19:13:44,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:13:44,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d6d602375b4506dfbd24 HTTP/1.1" 200 None 2026-03-29 19:13:44,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:13:44,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 19:13:44,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:13:44,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d6d602375b4506dfbd24 HTTP/1.1" 200 None 2026-03-29 19:13:44,492 - lava_callback - ERROR - Error processing callback for node 69c6d6d602375b4506dfbd24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56400 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:16:51,102 - lava_callback - INFO - Received callback for node 69c6d79102375b4506dfbd38 INFO: 172.20.0.1:37576 - "POST /node/69c6d79102375b4506dfbd38 HTTP/1.0" 202 Accepted 2026-03-29 19:16:51,108 - lava_callback - INFO - Starting async job submit for node 69c6d79102375b4506dfbd38 2026-03-29 19:16:51,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:16:51,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d79102375b4506dfbd38 HTTP/1.1" 200 None 2026-03-29 19:16:51,133 - lava_callback - DEBUG - Processing node 69c6d79102375b4506dfbd38: baseline-arm64-clabbe 2026-03-29 19:16:51,134 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d79102375b4506dfbd38 2026-03-29 19:16:51,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:16:51,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d79102375b4506dfbd38 HTTP/1.1" 200 None 2026-03-29 19:16:51,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:16:51,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 19:16:51,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:16:51,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d79102375b4506dfbd38 HTTP/1.1" 200 None 2026-03-29 19:16:51,249 - lava_callback - ERROR - Error processing callback for node 69c6d79102375b4506dfbd38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37588 - "GET /node/69876b21e31b58f834f19799%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:18:11,103 - lava_callback - INFO - Received callback for node 69c6d7df02375b4506dfbd40 INFO: 172.20.0.1:45568 - "POST /node/69c6d7df02375b4506dfbd40 HTTP/1.0" 202 Accepted 2026-03-29 19:18:11,109 - lava_callback - INFO - Starting async job submit for node 69c6d7df02375b4506dfbd40 2026-03-29 19:18:11,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:18:11,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d7df02375b4506dfbd40 HTTP/1.1" 200 None 2026-03-29 19:18:11,133 - lava_callback - DEBUG - Processing node 69c6d7df02375b4506dfbd40: baseline-arm64-clabbe 2026-03-29 19:18:11,134 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d7df02375b4506dfbd40 2026-03-29 19:18:11,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:18:11,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d7df02375b4506dfbd40 HTTP/1.1" 200 None 2026-03-29 19:18:11,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:18:11,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 19:18:11,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:18:11,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d7df02375b4506dfbd40 HTTP/1.1" 200 None 2026-03-29 19:18:11,190 - lava_callback - ERROR - Error processing callback for node 69c6d7df02375b4506dfbd40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38362 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:20:31,720 - lava_callback - INFO - Received callback for node 69c97242f2866272ea51e9d3 INFO: 172.20.0.1:38384 - "POST /node/69c97242f2866272ea51e9d3 HTTP/1.0" 202 Accepted 2026-03-29 19:20:31,729 - lava_callback - INFO - Starting async job submit for node 69c97242f2866272ea51e9d3 2026-03-29 19:20:31,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:31,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97242f2866272ea51e9d3 HTTP/1.1" 200 None 2026-03-29 19:20:31,819 - lava_callback - DEBUG - Processing node 69c97242f2866272ea51e9d3: baseline-nfs-arm64 2026-03-29 19:20:32,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 19:20:33,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 19:20:33,262 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c97242f2866272ea51e9d3/log.txt.gz 2026-03-29 19:20:33,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 19:20:34,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 19:20:34,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c97242f2866272ea51e9d3/lava-logs.yaml 2026-03-29 19:20:34,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 19:20:34,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 19:20:34,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c97242f2866272ea51e9d3/lava_callback.json.gz DEBUG: Setup failed for 69c97242f2866272ea51e9d3. Transitting job node result: incomplete -> fail 2026-03-29 19:20:34,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:34,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97242f2866272ea51e9d3 HTTP/1.1" 200 None 2026-03-29 19:20:34,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:34,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-29 19:20:34,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:34,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c97242f2866272ea51e9d3 HTTP/1.1" 200 None 2026-03-29 19:20:34,644 - lava_callback - INFO - Completed processing callback for node 69c97242f2866272ea51e9d3 INFO: 172.20.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53288 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:20:51,232 - lava_callback - INFO - Received callback for node 69c6d88202375b4506dfbd56 INFO: 172.20.0.1:49292 - "POST /node/69c6d88202375b4506dfbd56 HTTP/1.0" 202 Accepted 2026-03-29 19:20:51,238 - lava_callback - INFO - Starting async job submit for node 69c6d88202375b4506dfbd56 2026-03-29 19:20:51,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:51,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d88202375b4506dfbd56 HTTP/1.1" 200 None 2026-03-29 19:20:51,254 - lava_callback - DEBUG - Processing node 69c6d88202375b4506dfbd56: baseline-arm64-clabbe 2026-03-29 19:20:51,254 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d88202375b4506dfbd56 2026-03-29 19:20:51,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:51,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d88202375b4506dfbd56 HTTP/1.1" 200 None 2026-03-29 19:20:51,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:51,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 19:20:51,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:20:51,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d88202375b4506dfbd56 HTTP/1.1" 200 None 2026-03-29 19:20:51,320 - lava_callback - ERROR - Error processing callback for node 69c6d88202375b4506dfbd56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49296 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:21:00,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:21:00,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58028 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:22:11,298 - lava_callback - INFO - Received callback for node 69c6d8d202375b4506dfbd6a INFO: 172.20.0.1:60994 - "POST /node/69c6d8d202375b4506dfbd6a HTTP/1.0" 202 Accepted 2026-03-29 19:22:11,304 - lava_callback - INFO - Starting async job submit for node 69c6d8d202375b4506dfbd6a 2026-03-29 19:22:11,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:22:11,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d8d202375b4506dfbd6a HTTP/1.1" 200 None 2026-03-29 19:22:11,320 - lava_callback - DEBUG - Processing node 69c6d8d202375b4506dfbd6a: baseline-arm64-clabbe 2026-03-29 19:22:11,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d8d202375b4506dfbd6a 2026-03-29 19:22:11,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:22:11,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d8d202375b4506dfbd6a HTTP/1.1" 200 None 2026-03-29 19:22:11,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:22:11,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 19:22:11,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:22:11,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d8d202375b4506dfbd6a HTTP/1.1" 200 None 2026-03-29 19:22:11,388 - lava_callback - ERROR - Error processing callback for node 69c6d8d202375b4506dfbd6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51460 - "GET /node/693b845441fb4c950976596e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51484 - "GET /node/694b011a6c96ed51d3172ece%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:23:51,413 - lava_callback - INFO - Received callback for node 69c6d93502375b4506dfbd7e INFO: 172.20.0.1:56382 - "POST /node/69c6d93502375b4506dfbd7e HTTP/1.0" 202 Accepted 2026-03-29 19:23:51,419 - lava_callback - INFO - Starting async job submit for node 69c6d93502375b4506dfbd7e 2026-03-29 19:23:51,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:23:51,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d93502375b4506dfbd7e HTTP/1.1" 200 None 2026-03-29 19:23:51,435 - lava_callback - DEBUG - Processing node 69c6d93502375b4506dfbd7e: baseline-arm64-clabbe 2026-03-29 19:23:51,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d93502375b4506dfbd7e 2026-03-29 19:23:51,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:23:51,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d93502375b4506dfbd7e HTTP/1.1" 200 None 2026-03-29 19:23:51,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:23:51,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 19:23:51,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:23:51,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d93502375b4506dfbd7e HTTP/1.1" 200 None 2026-03-29 19:23:51,507 - lava_callback - ERROR - Error processing callback for node 69c6d93502375b4506dfbd7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39240 - "GET / HTTP/1.0" 200 OK 2026-03-29 19:25:11,392 - lava_callback - INFO - Received callback for node 69c6d98402375b4506dfbd86 INFO: 172.20.0.1:51826 - "POST /node/69c6d98402375b4506dfbd86 HTTP/1.0" 202 Accepted 2026-03-29 19:25:11,397 - lava_callback - INFO - Starting async job submit for node 69c6d98402375b4506dfbd86 2026-03-29 19:25:11,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:25:11,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d98402375b4506dfbd86 HTTP/1.1" 200 None 2026-03-29 19:25:11,417 - lava_callback - DEBUG - Processing node 69c6d98402375b4506dfbd86: baseline-arm64-clabbe 2026-03-29 19:25:11,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6d98402375b4506dfbd86 2026-03-29 19:25:11,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:25:11,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6d98402375b4506dfbd86 HTTP/1.1" 200 None 2026-03-29 19:25:11,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:25:11,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 19:25:11,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 19:25:11,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6d98402375b4506dfbd86 HTTP/1.1" 200 None 2026-03-29 19:25:11,469 - lava_callback - ERROR - Error processing callback for node 69c6d98402375b4506dfbd86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51198 - "GET /node/69934d15523507f3806a5bea%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41068 - "GET /node/6994656d5ae459b9df3e2acd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41070 - "GET /node/693ae38bfee8383e92b64178%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49420 - "GET /node/69840345969566dcb0d2b818%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53626 - "GET /node/691a6c638dbc6bccc0a0bbf2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET /node/698b8d600a16aea5a0dc8dd9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57164 - "GET /node/6993c9413e9ba493077102b9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46416 - "GET /node/693e49d5407f32424eb5fd1b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46428 - "GET /node/698b680c0a16aea5a0dc764b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57906 - "GET /node/698e28e5952a14bf500e401b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET /node/69ae1ed98ef4424e487b3dfd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57948 - "GET /node/698309fd409235c29c0dd741%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37878 - "GET /node/6991f7c301ff1e8e0859d446%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:39:49,204 - lava_callback - INFO - Received callback for node 69c6eb0602375b4506dfbe13 INFO: 172.20.0.1:37742 - "POST /node/69c6eb0602375b4506dfbe13 HTTP/1.0" 202 Accepted 2026-03-29 20:39:49,209 - lava_callback - INFO - Starting async job submit for node 69c6eb0602375b4506dfbe13 2026-03-29 20:39:49,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:39:49,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb0602375b4506dfbe13 HTTP/1.1" 200 None 2026-03-29 20:39:49,226 - lava_callback - DEBUG - Processing node 69c6eb0602375b4506dfbe13: baseline-arm-baylibre 2026-03-29 20:39:49,226 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6eb0602375b4506dfbe13 2026-03-29 20:39:49,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:39:49,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb0602375b4506dfbe13 HTTP/1.1" 200 None 2026-03-29 20:39:49,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:39:49,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:39:49,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:39:49,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6eb0602375b4506dfbe13 HTTP/1.1" 200 None 2026-03-29 20:39:49,288 - lava_callback - ERROR - Error processing callback for node 69c6eb0602375b4506dfbe13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36532 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:40:29,146 - lava_callback - INFO - Received callback for node 69c6eb2a02375b4506dfbe1e INFO: 172.20.0.1:36540 - "POST /node/69c6eb2a02375b4506dfbe1e HTTP/1.0" 202 Accepted 2026-03-29 20:40:29,152 - lava_callback - INFO - Starting async job submit for node 69c6eb2a02375b4506dfbe1e 2026-03-29 20:40:29,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:29,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb2a02375b4506dfbe1e HTTP/1.1" 200 None 2026-03-29 20:40:29,173 - lava_callback - DEBUG - Processing node 69c6eb2a02375b4506dfbe1e: baseline-arm-baylibre 2026-03-29 20:40:29,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6eb2a02375b4506dfbe1e 2026-03-29 20:40:29,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:29,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb2a02375b4506dfbe1e HTTP/1.1" 200 None 2026-03-29 20:40:29,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:29,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:40:29,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:29,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6eb2a02375b4506dfbe1e HTTP/1.1" 200 None 2026-03-29 20:40:29,226 - lava_callback - ERROR - Error processing callback for node 69c6eb2a02375b4506dfbe1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33798 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:40:49,136 - lava_callback - INFO - Received callback for node 69c6eb4f02375b4506dfbe2c INFO: 172.20.0.1:33804 - "POST /node/69c6eb4f02375b4506dfbe2c HTTP/1.0" 202 Accepted 2026-03-29 20:40:49,142 - lava_callback - INFO - Starting async job submit for node 69c6eb4f02375b4506dfbe2c 2026-03-29 20:40:49,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:49,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb4f02375b4506dfbe2c HTTP/1.1" 200 None 2026-03-29 20:40:49,160 - lava_callback - DEBUG - Processing node 69c6eb4f02375b4506dfbe2c: baseline-arm-baylibre 2026-03-29 20:40:49,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6eb4f02375b4506dfbe2c 2026-03-29 20:40:49,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:49,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb4f02375b4506dfbe2c HTTP/1.1" 200 None 2026-03-29 20:40:49,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:49,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:40:49,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:40:49,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6eb4f02375b4506dfbe2c HTTP/1.1" 200 None 2026-03-29 20:40:49,244 - lava_callback - ERROR - Error processing callback for node 69c6eb4f02375b4506dfbe2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44354 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:41:29,249 - lava_callback - INFO - Received callback for node 69c6eb7302375b4506dfbe3e INFO: 172.20.0.1:44370 - "POST /node/69c6eb7302375b4506dfbe3e HTTP/1.0" 202 Accepted 2026-03-29 20:41:29,255 - lava_callback - INFO - Starting async job submit for node 69c6eb7302375b4506dfbe3e 2026-03-29 20:41:29,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:41:29,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb7302375b4506dfbe3e HTTP/1.1" 200 None 2026-03-29 20:41:29,271 - lava_callback - DEBUG - Processing node 69c6eb7302375b4506dfbe3e: baseline-arm-baylibre 2026-03-29 20:41:29,271 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6eb7302375b4506dfbe3e 2026-03-29 20:41:29,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:41:29,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb7302375b4506dfbe3e HTTP/1.1" 200 None 2026-03-29 20:41:29,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:41:29,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:41:29,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:41:29,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6eb7302375b4506dfbe3e HTTP/1.1" 200 None 2026-03-29 20:41:29,339 - lava_callback - ERROR - Error processing callback for node 69c6eb7302375b4506dfbe3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33872 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:42:09,211 - lava_callback - INFO - Received callback for node 69c6eb9702375b4506dfbe4a INFO: 172.20.0.1:33878 - "POST /node/69c6eb9702375b4506dfbe4a HTTP/1.0" 202 Accepted 2026-03-29 20:42:09,217 - lava_callback - INFO - Starting async job submit for node 69c6eb9702375b4506dfbe4a 2026-03-29 20:42:09,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:09,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb9702375b4506dfbe4a HTTP/1.1" 200 None 2026-03-29 20:42:09,238 - lava_callback - DEBUG - Processing node 69c6eb9702375b4506dfbe4a: baseline-arm-baylibre 2026-03-29 20:42:09,239 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6eb9702375b4506dfbe4a 2026-03-29 20:42:09,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:09,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eb9702375b4506dfbe4a HTTP/1.1" 200 None 2026-03-29 20:42:09,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:09,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:42:09,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:09,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6eb9702375b4506dfbe4a HTTP/1.1" 200 None 2026-03-29 20:42:09,302 - lava_callback - ERROR - Error processing callback for node 69c6eb9702375b4506dfbe4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:42:49,303 - lava_callback - INFO - Received callback for node 69c6ebbb02375b4506dfbe5b INFO: 172.20.0.1:48584 - "POST /node/69c6ebbb02375b4506dfbe5b HTTP/1.0" 202 Accepted 2026-03-29 20:42:49,308 - lava_callback - INFO - Starting async job submit for node 69c6ebbb02375b4506dfbe5b 2026-03-29 20:42:49,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:49,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ebbb02375b4506dfbe5b HTTP/1.1" 200 None 2026-03-29 20:42:49,325 - lava_callback - DEBUG - Processing node 69c6ebbb02375b4506dfbe5b: baseline-arm-baylibre 2026-03-29 20:42:49,326 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ebbb02375b4506dfbe5b 2026-03-29 20:42:49,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:49,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ebbb02375b4506dfbe5b HTTP/1.1" 200 None 2026-03-29 20:42:49,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:49,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:42:49,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:42:49,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ebbb02375b4506dfbe5b HTTP/1.1" 200 None 2026-03-29 20:42:49,393 - lava_callback - ERROR - Error processing callback for node 69c6ebbb02375b4506dfbe5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60882 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:43:29,281 - lava_callback - INFO - Received callback for node 69c6ebe002375b4506dfbe68 INFO: 172.20.0.1:60886 - "POST /node/69c6ebe002375b4506dfbe68 HTTP/1.0" 202 Accepted 2026-03-29 20:43:29,286 - lava_callback - INFO - Starting async job submit for node 69c6ebe002375b4506dfbe68 2026-03-29 20:43:29,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:43:29,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ebe002375b4506dfbe68 HTTP/1.1" 200 None 2026-03-29 20:43:29,303 - lava_callback - DEBUG - Processing node 69c6ebe002375b4506dfbe68: baseline-arm-baylibre 2026-03-29 20:43:29,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ebe002375b4506dfbe68 2026-03-29 20:43:29,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:43:29,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ebe002375b4506dfbe68 HTTP/1.1" 200 None 2026-03-29 20:43:29,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:43:29,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:43:29,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:43:29,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ebe002375b4506dfbe68 HTTP/1.1" 200 None 2026-03-29 20:43:29,360 - lava_callback - ERROR - Error processing callback for node 69c6ebe002375b4506dfbe68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:44:09,373 - lava_callback - INFO - Received callback for node 69c6ec0402375b4506dfbe77 INFO: 172.20.0.1:58636 - "POST /node/69c6ec0402375b4506dfbe77 HTTP/1.0" 202 Accepted 2026-03-29 20:44:09,379 - lava_callback - INFO - Starting async job submit for node 69c6ec0402375b4506dfbe77 2026-03-29 20:44:09,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:09,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec0402375b4506dfbe77 HTTP/1.1" 200 None 2026-03-29 20:44:09,398 - lava_callback - DEBUG - Processing node 69c6ec0402375b4506dfbe77: baseline-arm-baylibre 2026-03-29 20:44:09,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ec0402375b4506dfbe77 2026-03-29 20:44:09,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:09,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec0402375b4506dfbe77 HTTP/1.1" 200 None 2026-03-29 20:44:09,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:09,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:44:09,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:09,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ec0402375b4506dfbe77 HTTP/1.1" 200 None 2026-03-29 20:44:09,466 - lava_callback - ERROR - Error processing callback for node 69c6ec0402375b4506dfbe77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36438 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:44:29,352 - lava_callback - INFO - Received callback for node 69c6ec2902375b4506dfbe89 INFO: 172.20.0.1:36446 - "POST /node/69c6ec2902375b4506dfbe89 HTTP/1.0" 202 Accepted 2026-03-29 20:44:29,358 - lava_callback - INFO - Starting async job submit for node 69c6ec2902375b4506dfbe89 2026-03-29 20:44:29,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:29,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec2902375b4506dfbe89 HTTP/1.1" 200 None 2026-03-29 20:44:29,375 - lava_callback - DEBUG - Processing node 69c6ec2902375b4506dfbe89: baseline-arm-baylibre 2026-03-29 20:44:29,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ec2902375b4506dfbe89 2026-03-29 20:44:29,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:29,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec2902375b4506dfbe89 HTTP/1.1" 200 None 2026-03-29 20:44:29,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:29,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:44:29,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:44:29,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ec2902375b4506dfbe89 HTTP/1.1" 200 None 2026-03-29 20:44:29,433 - lava_callback - ERROR - Error processing callback for node 69c6ec2902375b4506dfbe89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42548 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:45:09,418 - lava_callback - INFO - Received callback for node 69c6ec4d02375b4506dfbe99 INFO: 172.20.0.1:42560 - "POST /node/69c6ec4d02375b4506dfbe99 HTTP/1.0" 202 Accepted 2026-03-29 20:45:09,426 - lava_callback - INFO - Starting async job submit for node 69c6ec4d02375b4506dfbe99 2026-03-29 20:45:09,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:09,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec4d02375b4506dfbe99 HTTP/1.1" 200 None 2026-03-29 20:45:09,452 - lava_callback - DEBUG - Processing node 69c6ec4d02375b4506dfbe99: baseline-arm-baylibre 2026-03-29 20:45:09,453 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ec4d02375b4506dfbe99 2026-03-29 20:45:09,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:09,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec4d02375b4506dfbe99 HTTP/1.1" 200 None 2026-03-29 20:45:09,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:09,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:45:09,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:09,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ec4d02375b4506dfbe99 HTTP/1.1" 200 None 2026-03-29 20:45:09,509 - lava_callback - ERROR - Error processing callback for node 69c6ec4d02375b4506dfbe99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51520 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:45:49,490 - lava_callback - INFO - Received callback for node 69c6ec7102375b4506dfbeab INFO: 172.20.0.1:51522 - "POST /node/69c6ec7102375b4506dfbeab HTTP/1.0" 202 Accepted 2026-03-29 20:45:49,495 - lava_callback - INFO - Starting async job submit for node 69c6ec7102375b4506dfbeab 2026-03-29 20:45:49,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:49,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec7102375b4506dfbeab HTTP/1.1" 200 None 2026-03-29 20:45:49,512 - lava_callback - DEBUG - Processing node 69c6ec7102375b4506dfbeab: baseline-arm-baylibre 2026-03-29 20:45:49,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ec7102375b4506dfbeab 2026-03-29 20:45:49,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:49,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ec7102375b4506dfbeab HTTP/1.1" 200 None 2026-03-29 20:45:49,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:49,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:45:49,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:45:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ec7102375b4506dfbeab HTTP/1.1" 200 None 2026-03-29 20:45:49,579 - lava_callback - ERROR - Error processing callback for node 69c6ec7102375b4506dfbeab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET /node/699036b7985e4efca431cc3b%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-29 20:47:09,508 - lava_callback - INFO - Received callback for node 69c6ecb902375b4506dfbec8 INFO: 172.20.0.1:38664 - "POST /node/69c6ecb902375b4506dfbec8 HTTP/1.0" 202 Accepted 2026-03-29 20:47:09,514 - lava_callback - INFO - Starting async job submit for node 69c6ecb902375b4506dfbec8 2026-03-29 20:47:09,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:47:09,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ecb902375b4506dfbec8 HTTP/1.1" 200 None 2026-03-29 20:47:09,531 - lava_callback - DEBUG - Processing node 69c6ecb902375b4506dfbec8: baseline-arm-baylibre 2026-03-29 20:47:09,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ecb902375b4506dfbec8 2026-03-29 20:47:09,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:47:09,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ecb902375b4506dfbec8 HTTP/1.1" 200 None 2026-03-29 20:47:09,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:47:09,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:47:09,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:47:09,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ecb902375b4506dfbec8 HTTP/1.1" 200 None 2026-03-29 20:47:09,588 - lava_callback - ERROR - Error processing callback for node 69c6ecb902375b4506dfbec8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43572 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:48:14,953 - lava_callback - INFO - Received callback for node 69c6ed0402375b4506dfbeec INFO: 172.20.0.1:43574 - "POST /node/69c6ed0402375b4506dfbeec HTTP/1.0" 202 Accepted 2026-03-29 20:48:14,958 - lava_callback - INFO - Starting async job submit for node 69c6ed0402375b4506dfbeec 2026-03-29 20:48:14,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:14,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed0402375b4506dfbeec HTTP/1.1" 200 None 2026-03-29 20:48:14,975 - lava_callback - DEBUG - Processing node 69c6ed0402375b4506dfbeec: baseline-arm-clabbe 2026-03-29 20:48:14,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ed0402375b4506dfbeec 2026-03-29 20:48:14,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:14,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed0402375b4506dfbeec HTTP/1.1" 200 None 2026-03-29 20:48:14,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:15,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:48:15,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:15,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ed0402375b4506dfbeec HTTP/1.1" 200 None 2026-03-29 20:48:15,037 - lava_callback - ERROR - Error processing callback for node 69c6ed0402375b4506dfbeec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40540 - "GET /node/6980d4771ab488c16c858f81%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34648 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:48:54,920 - lava_callback - INFO - Received callback for node 69c6ed2902375b4506dfbefe INFO: 172.20.0.1:45798 - "POST /node/69c6ed2902375b4506dfbefe HTTP/1.0" 202 Accepted 2026-03-29 20:48:54,926 - lava_callback - INFO - Starting async job submit for node 69c6ed2902375b4506dfbefe 2026-03-29 20:48:54,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:54,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed2902375b4506dfbefe HTTP/1.1" 200 None 2026-03-29 20:48:54,973 - lava_callback - DEBUG - Processing node 69c6ed2902375b4506dfbefe: baseline-arm-clabbe 2026-03-29 20:48:54,974 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ed2902375b4506dfbefe 2026-03-29 20:48:54,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:54,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed2902375b4506dfbefe HTTP/1.1" 200 None 2026-03-29 20:48:54,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:55,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:48:55,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:48:55,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ed2902375b4506dfbefe HTTP/1.1" 200 None 2026-03-29 20:48:55,032 - lava_callback - ERROR - Error processing callback for node 69c6ed2902375b4506dfbefe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:49:34,986 - lava_callback - INFO - Received callback for node 69c6ed4e02375b4506dfbf0b INFO: 172.20.0.1:35010 - "POST /node/69c6ed4e02375b4506dfbf0b HTTP/1.0" 202 Accepted 2026-03-29 20:49:34,991 - lava_callback - INFO - Starting async job submit for node 69c6ed4e02375b4506dfbf0b 2026-03-29 20:49:34,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:49:35,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed4e02375b4506dfbf0b HTTP/1.1" 200 None 2026-03-29 20:49:35,008 - lava_callback - DEBUG - Processing node 69c6ed4e02375b4506dfbf0b: baseline-arm-clabbe 2026-03-29 20:49:35,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ed4e02375b4506dfbf0b 2026-03-29 20:49:35,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:49:35,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed4e02375b4506dfbf0b HTTP/1.1" 200 None 2026-03-29 20:49:35,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:49:35,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:49:35,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:49:35,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ed4e02375b4506dfbf0b HTTP/1.1" 200 None 2026-03-29 20:49:35,063 - lava_callback - ERROR - Error processing callback for node 69c6ed4e02375b4506dfbf0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55922 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:50:14,906 - lava_callback - INFO - Received callback for node 69c6ed7302375b4506dfbf1b INFO: 172.20.0.1:55930 - "POST /node/69c6ed7302375b4506dfbf1b HTTP/1.0" 202 Accepted 2026-03-29 20:50:14,913 - lava_callback - INFO - Starting async job submit for node 69c6ed7302375b4506dfbf1b 2026-03-29 20:50:14,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:14,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed7302375b4506dfbf1b HTTP/1.1" 200 None 2026-03-29 20:50:14,930 - lava_callback - DEBUG - Processing node 69c6ed7302375b4506dfbf1b: baseline-arm-clabbe 2026-03-29 20:50:14,930 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ed7302375b4506dfbf1b 2026-03-29 20:50:14,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:14,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed7302375b4506dfbf1b HTTP/1.1" 200 None 2026-03-29 20:50:14,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:14,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:50:14,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:14,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ed7302375b4506dfbf1b HTTP/1.1" 200 None 2026-03-29 20:50:14,986 - lava_callback - ERROR - Error processing callback for node 69c6ed7302375b4506dfbf1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:50:55,357 - lava_callback - INFO - Received callback for node 69c6ed9902375b4506dfbf33 INFO: 172.20.0.1:57218 - "POST /node/69c6ed9902375b4506dfbf33 HTTP/1.0" 202 Accepted 2026-03-29 20:50:55,363 - lava_callback - INFO - Starting async job submit for node 69c6ed9902375b4506dfbf33 2026-03-29 20:50:55,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:55,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed9902375b4506dfbf33 HTTP/1.1" 200 None 2026-03-29 20:50:55,379 - lava_callback - DEBUG - Processing node 69c6ed9902375b4506dfbf33: baseline-arm-clabbe 2026-03-29 20:50:55,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ed9902375b4506dfbf33 2026-03-29 20:50:55,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:55,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ed9902375b4506dfbf33 HTTP/1.1" 200 None 2026-03-29 20:50:55,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:55,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:50:55,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:50:55,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ed9902375b4506dfbf33 HTTP/1.1" 200 None 2026-03-29 20:50:55,442 - lava_callback - ERROR - Error processing callback for node 69c6ed9902375b4506dfbf33: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50180 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:51:15,027 - lava_callback - INFO - Received callback for node 69c6edbf02375b4506dfbf3e INFO: 172.20.0.1:50182 - "POST /node/69c6edbf02375b4506dfbf3e HTTP/1.0" 202 Accepted 2026-03-29 20:51:15,032 - lava_callback - INFO - Starting async job submit for node 69c6edbf02375b4506dfbf3e 2026-03-29 20:51:15,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:15,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6edbf02375b4506dfbf3e HTTP/1.1" 200 None 2026-03-29 20:51:15,049 - lava_callback - DEBUG - Processing node 69c6edbf02375b4506dfbf3e: baseline-arm-clabbe 2026-03-29 20:51:15,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6edbf02375b4506dfbf3e 2026-03-29 20:51:15,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:15,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6edbf02375b4506dfbf3e HTTP/1.1" 200 None 2026-03-29 20:51:15,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:15,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:51:15,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:15,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6edbf02375b4506dfbf3e HTTP/1.1" 200 None 2026-03-29 20:51:15,108 - lava_callback - ERROR - Error processing callback for node 69c6edbf02375b4506dfbf3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33852 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:51:55,062 - lava_callback - INFO - Received callback for node 69c6ede402375b4506dfbf51 INFO: 172.20.0.1:56898 - "POST /node/69c6ede402375b4506dfbf51 HTTP/1.0" 202 Accepted 2026-03-29 20:51:55,071 - lava_callback - INFO - Starting async job submit for node 69c6ede402375b4506dfbf51 2026-03-29 20:51:55,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:55,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ede402375b4506dfbf51 HTTP/1.1" 200 None 2026-03-29 20:51:55,087 - lava_callback - DEBUG - Processing node 69c6ede402375b4506dfbf51: baseline-arm-clabbe 2026-03-29 20:51:55,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ede402375b4506dfbf51 2026-03-29 20:51:55,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:55,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ede402375b4506dfbf51 HTTP/1.1" 200 None 2026-03-29 20:51:55,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:55,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:51:55,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:51:55,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ede402375b4506dfbf51 HTTP/1.1" 200 None 2026-03-29 20:51:55,171 - lava_callback - ERROR - Error processing callback for node 69c6ede402375b4506dfbf51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58128 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:52:35,048 - lava_callback - INFO - Received callback for node 69c6ee0a02375b4506dfbf61 INFO: 172.20.0.1:55034 - "POST /node/69c6ee0a02375b4506dfbf61 HTTP/1.0" 202 Accepted 2026-03-29 20:52:35,054 - lava_callback - INFO - Starting async job submit for node 69c6ee0a02375b4506dfbf61 2026-03-29 20:52:35,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:52:35,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ee0a02375b4506dfbf61 HTTP/1.1" 200 None 2026-03-29 20:52:35,070 - lava_callback - DEBUG - Processing node 69c6ee0a02375b4506dfbf61: baseline-arm-clabbe 2026-03-29 20:52:35,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ee0a02375b4506dfbf61 2026-03-29 20:52:35,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:52:35,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ee0a02375b4506dfbf61 HTTP/1.1" 200 None 2026-03-29 20:52:35,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:52:35,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-29 20:52:35,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:52:35,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ee0a02375b4506dfbf61 HTTP/1.1" 200 None 2026-03-29 20:52:35,132 - lava_callback - ERROR - Error processing callback for node 69c6ee0a02375b4506dfbf61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50906 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:53:15,101 - lava_callback - INFO - Received callback for node 69c6ee3002375b4506dfbf70 INFO: 172.20.0.1:50922 - "POST /node/69c6ee3002375b4506dfbf70 HTTP/1.0" 202 Accepted 2026-03-29 20:53:15,109 - lava_callback - INFO - Starting async job submit for node 69c6ee3002375b4506dfbf70 2026-03-29 20:53:15,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:53:15,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ee3002375b4506dfbf70 HTTP/1.1" 200 None 2026-03-29 20:53:15,127 - lava_callback - DEBUG - Processing node 69c6ee3002375b4506dfbf70: baseline-arm-clabbe 2026-03-29 20:53:15,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ee3002375b4506dfbf70 2026-03-29 20:53:15,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:53:15,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ee3002375b4506dfbf70 HTTP/1.1" 200 None 2026-03-29 20:53:15,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:53:15,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:53:15,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:50938 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 20:53:15,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ee3002375b4506dfbf70 HTTP/1.1" 200 None 2026-03-29 20:53:15,181 - lava_callback - ERROR - Error processing callback for node 69c6ee3002375b4506dfbf70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48398 - "GET /node/693fe90f575bc060ce14e73e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48400 - "GET /node/693a1fce78d7681e6b491595%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35216 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:54:55,157 - lava_callback - INFO - Received callback for node 69c6ee9a02375b4506dfbf99 INFO: 172.20.0.1:52016 - "POST /node/69c6ee9a02375b4506dfbf99 HTTP/1.0" 202 Accepted 2026-03-29 20:54:55,164 - lava_callback - INFO - Starting async job submit for node 69c6ee9a02375b4506dfbf99 2026-03-29 20:54:55,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:54:55,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ee9a02375b4506dfbf99 HTTP/1.1" 200 None 2026-03-29 20:54:55,189 - lava_callback - DEBUG - Processing node 69c6ee9a02375b4506dfbf99: baseline-arm-clabbe 2026-03-29 20:54:55,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ee9a02375b4506dfbf99 2026-03-29 20:54:55,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:54:55,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ee9a02375b4506dfbf99 HTTP/1.1" 200 None 2026-03-29 20:54:55,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:54:55,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:54:55,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:54:55,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ee9a02375b4506dfbf99 HTTP/1.1" 200 None 2026-03-29 20:54:55,240 - lava_callback - ERROR - Error processing callback for node 69c6ee9a02375b4506dfbf99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58664 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:56:10,085 - lava_callback - INFO - Received callback for node 69c6eee002375b4506dfbfb4 INFO: 172.20.0.1:36340 - "POST /node/69c6eee002375b4506dfbfb4 HTTP/1.0" 202 Accepted 2026-03-29 20:56:10,091 - lava_callback - INFO - Starting async job submit for node 69c6eee002375b4506dfbfb4 2026-03-29 20:56:10,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:10,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eee002375b4506dfbfb4 HTTP/1.1" 200 None 2026-03-29 20:56:10,107 - lava_callback - DEBUG - Processing node 69c6eee002375b4506dfbfb4: baseline-arm-baylibre 2026-03-29 20:56:10,107 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6eee002375b4506dfbfb4 2026-03-29 20:56:10,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:10,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6eee002375b4506dfbfb4 HTTP/1.1" 200 None 2026-03-29 20:56:10,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:10,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:56:10,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:10,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6eee002375b4506dfbfb4 HTTP/1.1" 200 None 2026-03-29 20:56:10,162 - lava_callback - ERROR - Error processing callback for node 69c6eee002375b4506dfbfb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:56:50,057 - lava_callback - INFO - Received callback for node 69c6ef0502375b4506dfbfc5 INFO: 172.20.0.1:45552 - "POST /node/69c6ef0502375b4506dfbfc5 HTTP/1.0" 202 Accepted 2026-03-29 20:56:50,064 - lava_callback - INFO - Starting async job submit for node 69c6ef0502375b4506dfbfc5 2026-03-29 20:56:50,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:50,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ef0502375b4506dfbfc5 HTTP/1.1" 200 None 2026-03-29 20:56:50,081 - lava_callback - DEBUG - Processing node 69c6ef0502375b4506dfbfc5: baseline-arm-baylibre 2026-03-29 20:56:50,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ef0502375b4506dfbfc5 2026-03-29 20:56:50,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:50,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ef0502375b4506dfbfc5 HTTP/1.1" 200 None 2026-03-29 20:56:50,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:50,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:56:50,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:56:50,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ef0502375b4506dfbfc5 HTTP/1.1" 200 None 2026-03-29 20:56:50,162 - lava_callback - ERROR - Error processing callback for node 69c6ef0502375b4506dfbfc5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK 2026-03-29 20:58:35,291 - lava_callback - INFO - Received callback for node 69c6ef7102375b4506dfbff0 INFO: 172.20.0.1:46538 - "POST /node/69c6ef7102375b4506dfbff0 HTTP/1.0" 202 Accepted 2026-03-29 20:58:35,296 - lava_callback - INFO - Starting async job submit for node 69c6ef7102375b4506dfbff0 2026-03-29 20:58:35,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:58:35,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ef7102375b4506dfbff0 HTTP/1.1" 200 None 2026-03-29 20:58:35,330 - lava_callback - DEBUG - Processing node 69c6ef7102375b4506dfbff0: baseline-arm-clabbe 2026-03-29 20:58:35,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ef7102375b4506dfbff0 2026-03-29 20:58:35,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:58:35,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ef7102375b4506dfbff0 HTTP/1.1" 200 None 2026-03-29 20:58:35,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:58:35,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 20:58:35,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 20:58:35,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ef7102375b4506dfbff0 HTTP/1.1" 200 None 2026-03-29 20:58:35,384 - lava_callback - ERROR - Error processing callback for node 69c6ef7102375b4506dfbff0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET /node/69c17c172b2b51e9ad97a7e8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 21:00:15,542 - lava_callback - INFO - Received callback for node 69c6efdc02375b4506dfc012 INFO: 172.20.0.1:55640 - "POST /node/69c6efdc02375b4506dfc012 HTTP/1.0" 202 Accepted 2026-03-29 21:00:15,548 - lava_callback - INFO - Starting async job submit for node 69c6efdc02375b4506dfc012 2026-03-29 21:00:15,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:15,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6efdc02375b4506dfc012 HTTP/1.1" 200 None 2026-03-29 21:00:15,564 - lava_callback - DEBUG - Processing node 69c6efdc02375b4506dfc012: baseline-arm-clabbe 2026-03-29 21:00:15,564 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6efdc02375b4506dfc012 2026-03-29 21:00:15,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:15,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6efdc02375b4506dfc012 HTTP/1.1" 200 None 2026-03-29 21:00:15,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:15,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:00:15,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:15,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6efdc02375b4506dfc012 HTTP/1.1" 200 None 2026-03-29 21:00:15,623 - lava_callback - ERROR - Error processing callback for node 69c6efdc02375b4506dfc012: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32968 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:00:55,372 - lava_callback - INFO - Received callback for node 69c6f00102375b4506dfc01e INFO: 172.20.0.1:51228 - "POST /node/69c6f00102375b4506dfc01e HTTP/1.0" 202 Accepted 2026-03-29 21:00:55,377 - lava_callback - INFO - Starting async job submit for node 69c6f00102375b4506dfc01e 2026-03-29 21:00:55,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:55,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f00102375b4506dfc01e HTTP/1.1" 200 None 2026-03-29 21:00:55,397 - lava_callback - DEBUG - Processing node 69c6f00102375b4506dfc01e: baseline-arm-clabbe 2026-03-29 21:00:55,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f00102375b4506dfc01e 2026-03-29 21:00:55,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:55,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f00102375b4506dfc01e HTTP/1.1" 200 None 2026-03-29 21:00:55,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:55,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:00:55,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:00:55,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f00102375b4506dfc01e HTTP/1.1" 200 None 2026-03-29 21:00:55,477 - lava_callback - ERROR - Error processing callback for node 69c6f00102375b4506dfc01e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:01:35,496 - lava_callback - INFO - Received callback for node 69c6f02702375b4506dfc02a INFO: 172.20.0.1:43384 - "POST /node/69c6f02702375b4506dfc02a HTTP/1.0" 202 Accepted 2026-03-29 21:01:35,501 - lava_callback - INFO - Starting async job submit for node 69c6f02702375b4506dfc02a 2026-03-29 21:01:35,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:01:35,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f02702375b4506dfc02a HTTP/1.1" 200 None 2026-03-29 21:01:35,517 - lava_callback - DEBUG - Processing node 69c6f02702375b4506dfc02a: baseline-arm-clabbe 2026-03-29 21:01:35,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f02702375b4506dfc02a 2026-03-29 21:01:35,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:01:35,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f02702375b4506dfc02a HTTP/1.1" 200 None 2026-03-29 21:01:35,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:01:35,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:01:35,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:01:35,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f02702375b4506dfc02a HTTP/1.1" 200 None 2026-03-29 21:01:35,574 - lava_callback - ERROR - Error processing callback for node 69c6f02702375b4506dfc02a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51554 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 21:02:15,419 - lava_callback - INFO - Received callback for node 69c6f04c02375b4506dfc037 INFO: 172.20.0.1:51566 - "POST /node/69c6f04c02375b4506dfc037 HTTP/1.0" 202 Accepted 2026-03-29 21:02:15,425 - lava_callback - INFO - Starting async job submit for node 69c6f04c02375b4506dfc037 2026-03-29 21:02:15,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:15,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f04c02375b4506dfc037 HTTP/1.1" 200 None 2026-03-29 21:02:15,447 - lava_callback - DEBUG - Processing node 69c6f04c02375b4506dfc037: baseline-arm-clabbe 2026-03-29 21:02:15,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f04c02375b4506dfc037 2026-03-29 21:02:15,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:15,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f04c02375b4506dfc037 HTTP/1.1" 200 None 2026-03-29 21:02:15,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:15,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:02:15,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:15,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f04c02375b4506dfc037 HTTP/1.1" 200 None 2026-03-29 21:02:15,504 - lava_callback - ERROR - Error processing callback for node 69c6f04c02375b4506dfc037: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50246 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:02:55,607 - lava_callback - INFO - Received callback for node 69c6f07102375b4506dfc03e INFO: 172.20.0.1:35198 - "POST /node/69c6f07102375b4506dfc03e HTTP/1.0" 202 Accepted 2026-03-29 21:02:55,613 - lava_callback - INFO - Starting async job submit for node 69c6f07102375b4506dfc03e 2026-03-29 21:02:55,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:55,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f07102375b4506dfc03e HTTP/1.1" 200 None 2026-03-29 21:02:55,633 - lava_callback - DEBUG - Processing node 69c6f07102375b4506dfc03e: baseline-arm-clabbe 2026-03-29 21:02:55,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f07102375b4506dfc03e 2026-03-29 21:02:55,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:55,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f07102375b4506dfc03e HTTP/1.1" 200 None 2026-03-29 21:02:55,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:55,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:02:55,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:02:55,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f07102375b4506dfc03e HTTP/1.1" 200 None 2026-03-29 21:02:55,746 - lava_callback - ERROR - Error processing callback for node 69c6f07102375b4506dfc03e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:03:35,460 - lava_callback - INFO - Received callback for node 69c6f09602375b4506dfc047 INFO: 172.20.0.1:51542 - "POST /node/69c6f09602375b4506dfc047 HTTP/1.0" 202 Accepted 2026-03-29 21:03:35,466 - lava_callback - INFO - Starting async job submit for node 69c6f09602375b4506dfc047 2026-03-29 21:03:35,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:03:35,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f09602375b4506dfc047 HTTP/1.1" 200 None 2026-03-29 21:03:35,482 - lava_callback - DEBUG - Processing node 69c6f09602375b4506dfc047: baseline-arm-clabbe 2026-03-29 21:03:35,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f09602375b4506dfc047 2026-03-29 21:03:35,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:03:35,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f09602375b4506dfc047 HTTP/1.1" 200 None 2026-03-29 21:03:35,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:03:35,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:03:35,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:03:35,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f09602375b4506dfc047 HTTP/1.1" 200 None 2026-03-29 21:03:35,550 - lava_callback - ERROR - Error processing callback for node 69c6f09602375b4506dfc047: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47042 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 21:04:15,494 - lava_callback - INFO - Received callback for node 69c6f0ba02375b4506dfc050 INFO: 172.20.0.1:47044 - "POST /node/69c6f0ba02375b4506dfc050 HTTP/1.0" 202 Accepted 2026-03-29 21:04:15,499 - lava_callback - INFO - Starting async job submit for node 69c6f0ba02375b4506dfc050 2026-03-29 21:04:15,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:15,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f0ba02375b4506dfc050 HTTP/1.1" 200 None 2026-03-29 21:04:15,516 - lava_callback - DEBUG - Processing node 69c6f0ba02375b4506dfc050: baseline-arm-clabbe 2026-03-29 21:04:15,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f0ba02375b4506dfc050 2026-03-29 21:04:15,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:15,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f0ba02375b4506dfc050 HTTP/1.1" 200 None 2026-03-29 21:04:15,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:15,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:04:15,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:15,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f0ba02375b4506dfc050 HTTP/1.1" 200 None 2026-03-29 21:04:15,578 - lava_callback - ERROR - Error processing callback for node 69c6f0ba02375b4506dfc050: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:04:35,465 - lava_callback - INFO - Received callback for node 69c6f0de02375b4506dfc05b INFO: 172.20.0.1:40688 - "POST /node/69c6f0de02375b4506dfc05b HTTP/1.0" 202 Accepted 2026-03-29 21:04:35,471 - lava_callback - INFO - Starting async job submit for node 69c6f0de02375b4506dfc05b 2026-03-29 21:04:35,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:35,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f0de02375b4506dfc05b HTTP/1.1" 200 None 2026-03-29 21:04:35,486 - lava_callback - DEBUG - Processing node 69c6f0de02375b4506dfc05b: baseline-arm-clabbe 2026-03-29 21:04:35,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f0de02375b4506dfc05b 2026-03-29 21:04:35,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:35,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f0de02375b4506dfc05b HTTP/1.1" 200 None 2026-03-29 21:04:35,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:35,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:04:35,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:04:35,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f0de02375b4506dfc05b HTTP/1.1" 200 None 2026-03-29 21:04:35,547 - lava_callback - ERROR - Error processing callback for node 69c6f0de02375b4506dfc05b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55188 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 21:05:15,511 - lava_callback - INFO - Received callback for node 69c6f10202375b4506dfc062 INFO: 172.20.0.1:55192 - "POST /node/69c6f10202375b4506dfc062 HTTP/1.0" 202 Accepted 2026-03-29 21:05:15,519 - lava_callback - INFO - Starting async job submit for node 69c6f10202375b4506dfc062 2026-03-29 21:05:15,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:05:15,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f10202375b4506dfc062 HTTP/1.1" 200 None 2026-03-29 21:05:15,534 - lava_callback - DEBUG - Processing node 69c6f10202375b4506dfc062: baseline-arm-clabbe 2026-03-29 21:05:15,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f10202375b4506dfc062 2026-03-29 21:05:15,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:05:15,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f10202375b4506dfc062 HTTP/1.1" 200 None 2026-03-29 21:05:15,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:05:15,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-29 21:05:15,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:05:15,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f10202375b4506dfc062 HTTP/1.1" 200 None 2026-03-29 21:05:15,610 - lava_callback - ERROR - Error processing callback for node 69c6f10202375b4506dfc062: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49978 - "GET /node/698464d13a8748f282086a15%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54952 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:10:50,867 - lava_callback - INFO - Received callback for node 69c6f24c02375b4506dfc074 INFO: 172.20.0.1:55298 - "POST /node/69c6f24c02375b4506dfc074 HTTP/1.0" 202 Accepted 2026-03-29 21:10:50,873 - lava_callback - INFO - Starting async job submit for node 69c6f24c02375b4506dfc074 2026-03-29 21:10:50,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:10:50,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f24c02375b4506dfc074 HTTP/1.1" 200 None 2026-03-29 21:10:50,889 - lava_callback - DEBUG - Processing node 69c6f24c02375b4506dfc074: baseline-arm64-baylibre 2026-03-29 21:10:50,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f24c02375b4506dfc074 2026-03-29 21:10:50,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:10:50,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f24c02375b4506dfc074 HTTP/1.1" 200 None 2026-03-29 21:10:50,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:10:50,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:10:50,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:10:50,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f24c02375b4506dfc074 HTTP/1.1" 200 None 2026-03-29 21:10:50,946 - lava_callback - ERROR - Error processing callback for node 69c6f24c02375b4506dfc074: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:17:31,251 - lava_callback - INFO - Received callback for node 69c6f3d802375b4506dfc0ae INFO: 172.20.0.1:39528 - "POST /node/69c6f3d802375b4506dfc0ae HTTP/1.0" 202 Accepted 2026-03-29 21:17:31,257 - lava_callback - INFO - Starting async job submit for node 69c6f3d802375b4506dfc0ae 2026-03-29 21:17:31,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:17:31,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f3d802375b4506dfc0ae HTTP/1.1" 200 None 2026-03-29 21:17:31,291 - lava_callback - DEBUG - Processing node 69c6f3d802375b4506dfc0ae: baseline-arm64-baylibre 2026-03-29 21:17:31,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f3d802375b4506dfc0ae 2026-03-29 21:17:31,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:17:31,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f3d802375b4506dfc0ae HTTP/1.1" 200 None 2026-03-29 21:17:31,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:17:31,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:17:31,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:17:31,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f3d802375b4506dfc0ae HTTP/1.1" 200 None 2026-03-29 21:17:31,349 - lava_callback - ERROR - Error processing callback for node 69c6f3d802375b4506dfc0ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43010 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:18:31,232 - lava_callback - INFO - Received callback for node 69c6f42502375b4506dfc0bd INFO: 172.20.0.1:49600 - "POST /node/69c6f42502375b4506dfc0bd HTTP/1.0" 202 Accepted 2026-03-29 21:18:31,238 - lava_callback - INFO - Starting async job submit for node 69c6f42502375b4506dfc0bd 2026-03-29 21:18:31,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:18:31,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f42502375b4506dfc0bd HTTP/1.1" 200 None 2026-03-29 21:18:31,254 - lava_callback - DEBUG - Processing node 69c6f42502375b4506dfc0bd: baseline-arm64-baylibre 2026-03-29 21:18:31,254 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f42502375b4506dfc0bd 2026-03-29 21:18:31,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:18:31,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f42502375b4506dfc0bd HTTP/1.1" 200 None 2026-03-29 21:18:31,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:18:31,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:18:31,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:18:31,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f42502375b4506dfc0bd HTTP/1.1" 200 None 2026-03-29 21:18:31,311 - lava_callback - ERROR - Error processing callback for node 69c6f42502375b4506dfc0bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54494 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:20:11,370 - lava_callback - INFO - Received callback for node 69c6f47602375b4506dfc0c6 INFO: 172.20.0.1:56566 - "POST /node/69c6f47602375b4506dfc0c6 HTTP/1.0" 202 Accepted 2026-03-29 21:20:11,376 - lava_callback - INFO - Starting async job submit for node 69c6f47602375b4506dfc0c6 2026-03-29 21:20:11,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:20:11,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f47602375b4506dfc0c6 HTTP/1.1" 200 None 2026-03-29 21:20:11,392 - lava_callback - DEBUG - Processing node 69c6f47602375b4506dfc0c6: baseline-arm64-baylibre 2026-03-29 21:20:11,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f47602375b4506dfc0c6 2026-03-29 21:20:11,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:20:11,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f47602375b4506dfc0c6 HTTP/1.1" 200 None 2026-03-29 21:20:11,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:20:11,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:20:11,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:20:11,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f47602375b4506dfc0c6 HTTP/1.1" 200 None 2026-03-29 21:20:11,458 - lava_callback - ERROR - Error processing callback for node 69c6f47602375b4506dfc0c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43312 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:21:31,412 - lava_callback - INFO - Received callback for node 69c6f4c602375b4506dfc0d2 INFO: 172.20.0.1:58766 - "POST /node/69c6f4c602375b4506dfc0d2 HTTP/1.0" 202 Accepted 2026-03-29 21:21:31,418 - lava_callback - INFO - Starting async job submit for node 69c6f4c602375b4506dfc0d2 2026-03-29 21:21:31,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:21:31,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f4c602375b4506dfc0d2 HTTP/1.1" 200 None 2026-03-29 21:21:31,434 - lava_callback - DEBUG - Processing node 69c6f4c602375b4506dfc0d2: baseline-arm64-baylibre 2026-03-29 21:21:31,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f4c602375b4506dfc0d2 2026-03-29 21:21:31,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:21:31,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f4c602375b4506dfc0d2 HTTP/1.1" 200 None 2026-03-29 21:21:31,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:21:31,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:21:31,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:21:31,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f4c602375b4506dfc0d2 HTTP/1.1" 200 None 2026-03-29 21:21:31,488 - lava_callback - ERROR - Error processing callback for node 69c6f4c602375b4506dfc0d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58964 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:22:51,613 - lava_callback - INFO - Received callback for node 69c6f51702375b4506dfc0e5 INFO: 172.20.0.1:35446 - "POST /node/69c6f51702375b4506dfc0e5 HTTP/1.0" 202 Accepted 2026-03-29 21:22:51,619 - lava_callback - INFO - Starting async job submit for node 69c6f51702375b4506dfc0e5 2026-03-29 21:22:51,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:22:51,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f51702375b4506dfc0e5 HTTP/1.1" 200 None 2026-03-29 21:22:51,635 - lava_callback - DEBUG - Processing node 69c6f51702375b4506dfc0e5: baseline-arm64-baylibre 2026-03-29 21:22:51,635 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f51702375b4506dfc0e5 2026-03-29 21:22:51,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:22:51,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f51702375b4506dfc0e5 HTTP/1.1" 200 None 2026-03-29 21:22:51,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:22:51,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:22:51,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:22:51,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f51702375b4506dfc0e5 HTTP/1.1" 200 None 2026-03-29 21:22:51,691 - lava_callback - ERROR - Error processing callback for node 69c6f51702375b4506dfc0e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49374 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:25:11,730 - lava_callback - INFO - Received callback for node 69c6f5ab02375b4506dfc0f6 INFO: 172.20.0.1:39200 - "POST /node/69c6f5ab02375b4506dfc0f6 HTTP/1.0" 202 Accepted 2026-03-29 21:25:11,736 - lava_callback - INFO - Starting async job submit for node 69c6f5ab02375b4506dfc0f6 2026-03-29 21:25:11,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:25:11,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f5ab02375b4506dfc0f6 HTTP/1.1" 200 None 2026-03-29 21:25:11,752 - lava_callback - DEBUG - Processing node 69c6f5ab02375b4506dfc0f6: baseline-arm64-baylibre 2026-03-29 21:25:11,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f5ab02375b4506dfc0f6 2026-03-29 21:25:11,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:25:11,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f5ab02375b4506dfc0f6 HTTP/1.1" 200 None 2026-03-29 21:25:11,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:25:11,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:25:11,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:25:11,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f5ab02375b4506dfc0f6 HTTP/1.1" 200 None 2026-03-29 21:25:11,812 - lava_callback - ERROR - Error processing callback for node 69c6f5ab02375b4506dfc0f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55228 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:26:31,846 - lava_callback - INFO - Received callback for node 69c6f5fb02375b4506dfc104 INFO: 172.20.0.1:60196 - "POST /node/69c6f5fb02375b4506dfc104 HTTP/1.0" 202 Accepted 2026-03-29 21:26:31,852 - lava_callback - INFO - Starting async job submit for node 69c6f5fb02375b4506dfc104 2026-03-29 21:26:31,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:26:31,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f5fb02375b4506dfc104 HTTP/1.1" 200 None 2026-03-29 21:26:31,868 - lava_callback - DEBUG - Processing node 69c6f5fb02375b4506dfc104: baseline-arm64-baylibre 2026-03-29 21:26:31,869 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f5fb02375b4506dfc104 2026-03-29 21:26:31,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:26:31,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f5fb02375b4506dfc104 HTTP/1.1" 200 None 2026-03-29 21:26:31,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:26:31,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:26:31,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:26:31,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f5fb02375b4506dfc104 HTTP/1.1" 200 None 2026-03-29 21:26:31,930 - lava_callback - ERROR - Error processing callback for node 69c6f5fb02375b4506dfc104: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45468 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:27:56,570 - lava_callback - INFO - Received callback for node 69c6f65102375b4506dfc115 INFO: 172.20.0.1:45482 - "POST /node/69c6f65102375b4506dfc115 HTTP/1.0" 202 Accepted 2026-03-29 21:27:56,576 - lava_callback - INFO - Starting async job submit for node 69c6f65102375b4506dfc115 2026-03-29 21:27:56,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:27:56,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f65102375b4506dfc115 HTTP/1.1" 200 None 2026-03-29 21:27:56,600 - lava_callback - DEBUG - Processing node 69c6f65102375b4506dfc115: baseline-arm64-clabbe 2026-03-29 21:27:56,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f65102375b4506dfc115 2026-03-29 21:27:56,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:27:56,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f65102375b4506dfc115 HTTP/1.1" 200 None 2026-03-29 21:27:56,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:27:56,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:27:56,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:27:56,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f65102375b4506dfc115 HTTP/1.1" 200 None 2026-03-29 21:27:56,657 - lava_callback - ERROR - Error processing callback for node 69c6f65102375b4506dfc115: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:29:11,834 - lava_callback - INFO - Received callback for node 69c6f69a02375b4506dfc11b INFO: 172.20.0.1:48676 - "POST /node/69c6f69a02375b4506dfc11b HTTP/1.0" 202 Accepted 2026-03-29 21:29:11,839 - lava_callback - INFO - Starting async job submit for node 69c6f69a02375b4506dfc11b 2026-03-29 21:29:11,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:29:11,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f69a02375b4506dfc11b HTTP/1.1" 200 None 2026-03-29 21:29:11,856 - lava_callback - DEBUG - Processing node 69c6f69a02375b4506dfc11b: baseline-arm64-baylibre 2026-03-29 21:29:11,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f69a02375b4506dfc11b 2026-03-29 21:29:11,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:29:11,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f69a02375b4506dfc11b HTTP/1.1" 200 None 2026-03-29 21:29:11,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:29:11,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:29:11,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:29:11,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f69a02375b4506dfc11b HTTP/1.1" 200 None 2026-03-29 21:29:11,914 - lava_callback - ERROR - Error processing callback for node 69c6f69a02375b4506dfc11b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46690 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:30:31,941 - lava_callback - INFO - Received callback for node 69c6f6e802375b4506dfc129 INFO: 172.20.0.1:53856 - "POST /node/69c6f6e802375b4506dfc129 HTTP/1.0" 202 Accepted 2026-03-29 21:30:31,947 - lava_callback - INFO - Starting async job submit for node 69c6f6e802375b4506dfc129 2026-03-29 21:30:31,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:30:31,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f6e802375b4506dfc129 HTTP/1.1" 200 None 2026-03-29 21:30:31,968 - lava_callback - DEBUG - Processing node 69c6f6e802375b4506dfc129: baseline-arm64-baylibre 2026-03-29 21:30:31,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f6e802375b4506dfc129 2026-03-29 21:30:31,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:30:31,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f6e802375b4506dfc129 HTTP/1.1" 200 None 2026-03-29 21:30:31,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:30:31,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:30:31,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:30:32,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f6e802375b4506dfc129 HTTP/1.1" 200 None 2026-03-29 21:30:32,021 - lava_callback - ERROR - Error processing callback for node 69c6f6e802375b4506dfc129: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54148 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:31:56,698 - lava_callback - INFO - Received callback for node 69c6f73e02375b4506dfc13c INFO: 172.20.0.1:54162 - "POST /node/69c6f73e02375b4506dfc13c HTTP/1.0" 202 Accepted 2026-03-29 21:31:56,704 - lava_callback - INFO - Starting async job submit for node 69c6f73e02375b4506dfc13c 2026-03-29 21:31:56,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:31:56,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f73e02375b4506dfc13c HTTP/1.1" 200 None 2026-03-29 21:31:56,720 - lava_callback - DEBUG - Processing node 69c6f73e02375b4506dfc13c: baseline-arm64-clabbe 2026-03-29 21:31:56,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f73e02375b4506dfc13c 2026-03-29 21:31:56,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:31:56,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f73e02375b4506dfc13c HTTP/1.1" 200 None 2026-03-29 21:31:56,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:31:56,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:31:56,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:31:56,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f73e02375b4506dfc13c HTTP/1.1" 200 None 2026-03-29 21:31:56,780 - lava_callback - ERROR - Error processing callback for node 69c6f73e02375b4506dfc13c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58792 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:33:16,726 - lava_callback - INFO - Received callback for node 69c6f78602375b4506dfc142 INFO: 172.20.0.1:58800 - "POST /node/69c6f78602375b4506dfc142 HTTP/1.0" 202 Accepted 2026-03-29 21:33:16,732 - lava_callback - INFO - Starting async job submit for node 69c6f78602375b4506dfc142 2026-03-29 21:33:16,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:33:16,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f78602375b4506dfc142 HTTP/1.1" 200 None 2026-03-29 21:33:16,747 - lava_callback - DEBUG - Processing node 69c6f78602375b4506dfc142: baseline-arm64-clabbe 2026-03-29 21:33:16,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f78602375b4506dfc142 2026-03-29 21:33:16,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:33:16,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f78602375b4506dfc142 HTTP/1.1" 200 None 2026-03-29 21:33:16,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:33:16,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:33:16,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:33:16,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f78602375b4506dfc142 HTTP/1.1" 200 None 2026-03-29 21:33:16,815 - lava_callback - ERROR - Error processing callback for node 69c6f78602375b4506dfc142: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59784 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:34:16,796 - lava_callback - INFO - Received callback for node 69c6f7d402375b4506dfc152 INFO: 172.20.0.1:59798 - "POST /node/69c6f7d402375b4506dfc152 HTTP/1.0" 202 Accepted 2026-03-29 21:34:16,802 - lava_callback - INFO - Starting async job submit for node 69c6f7d402375b4506dfc152 2026-03-29 21:34:16,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:34:16,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f7d402375b4506dfc152 HTTP/1.1" 200 None 2026-03-29 21:34:16,818 - lava_callback - DEBUG - Processing node 69c6f7d402375b4506dfc152: baseline-arm64-clabbe 2026-03-29 21:34:16,818 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f7d402375b4506dfc152 2026-03-29 21:34:16,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:34:16,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f7d402375b4506dfc152 HTTP/1.1" 200 None 2026-03-29 21:34:16,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:34:16,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:34:16,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:34:16,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f7d402375b4506dfc152 HTTP/1.1" 200 None 2026-03-29 21:34:16,873 - lava_callback - ERROR - Error processing callback for node 69c6f7d402375b4506dfc152: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:35:36,811 - lava_callback - INFO - Received callback for node 69c6f82202375b4506dfc15e INFO: 172.20.0.1:50302 - "POST /node/69c6f82202375b4506dfc15e HTTP/1.0" 202 Accepted 2026-03-29 21:35:36,817 - lava_callback - INFO - Starting async job submit for node 69c6f82202375b4506dfc15e 2026-03-29 21:35:36,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:35:36,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f82202375b4506dfc15e HTTP/1.1" 200 None 2026-03-29 21:35:36,833 - lava_callback - DEBUG - Processing node 69c6f82202375b4506dfc15e: baseline-arm64-clabbe 2026-03-29 21:35:36,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f82202375b4506dfc15e 2026-03-29 21:35:36,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:35:36,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f82202375b4506dfc15e HTTP/1.1" 200 None 2026-03-29 21:35:36,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:35:36,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:35:36,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:35:36,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f82202375b4506dfc15e HTTP/1.1" 200 None 2026-03-29 21:35:36,907 - lava_callback - ERROR - Error processing callback for node 69c6f82202375b4506dfc15e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59088 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:36:56,825 - lava_callback - INFO - Received callback for node 69c6f87102375b4506dfc168 INFO: 172.20.0.1:59096 - "POST /node/69c6f87102375b4506dfc168 HTTP/1.0" 202 Accepted 2026-03-29 21:36:56,831 - lava_callback - INFO - Starting async job submit for node 69c6f87102375b4506dfc168 2026-03-29 21:36:56,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:36:56,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f87102375b4506dfc168 HTTP/1.1" 200 None 2026-03-29 21:36:56,847 - lava_callback - DEBUG - Processing node 69c6f87102375b4506dfc168: baseline-arm64-clabbe 2026-03-29 21:36:56,848 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f87102375b4506dfc168 2026-03-29 21:36:56,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:36:56,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f87102375b4506dfc168 HTTP/1.1" 200 None 2026-03-29 21:36:56,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:36:56,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:36:56,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:36:56,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f87102375b4506dfc168 HTTP/1.1" 200 None 2026-03-29 21:36:56,899 - lava_callback - ERROR - Error processing callback for node 69c6f87102375b4506dfc168: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43114 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:38:16,885 - lava_callback - INFO - Received callback for node 69c6f8c002375b4506dfc174 INFO: 172.20.0.1:43120 - "POST /node/69c6f8c002375b4506dfc174 HTTP/1.0" 202 Accepted 2026-03-29 21:38:16,890 - lava_callback - INFO - Starting async job submit for node 69c6f8c002375b4506dfc174 2026-03-29 21:38:16,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:38:16,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f8c002375b4506dfc174 HTTP/1.1" 200 None 2026-03-29 21:38:16,907 - lava_callback - DEBUG - Processing node 69c6f8c002375b4506dfc174: baseline-arm64-clabbe 2026-03-29 21:38:16,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f8c002375b4506dfc174 2026-03-29 21:38:16,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:38:16,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f8c002375b4506dfc174 HTTP/1.1" 200 None 2026-03-29 21:38:16,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:38:16,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:38:16,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:38:16,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f8c002375b4506dfc174 HTTP/1.1" 200 None 2026-03-29 21:38:16,962 - lava_callback - ERROR - Error processing callback for node 69c6f8c002375b4506dfc174: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53992 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:39:36,939 - lava_callback - INFO - Received callback for node 69c6f90e02375b4506dfc17c INFO: 172.20.0.1:54008 - "POST /node/69c6f90e02375b4506dfc17c HTTP/1.0" 202 Accepted 2026-03-29 21:39:36,946 - lava_callback - INFO - Starting async job submit for node 69c6f90e02375b4506dfc17c 2026-03-29 21:39:36,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:39:37,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f90e02375b4506dfc17c HTTP/1.1" 200 None 2026-03-29 21:39:37,004 - lava_callback - DEBUG - Processing node 69c6f90e02375b4506dfc17c: baseline-arm64-clabbe 2026-03-29 21:39:37,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f90e02375b4506dfc17c 2026-03-29 21:39:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:39:37,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f90e02375b4506dfc17c HTTP/1.1" 200 None 2026-03-29 21:39:37,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:39:37,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:39:37,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:39:37,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f90e02375b4506dfc17c HTTP/1.1" 200 None 2026-03-29 21:39:37,080 - lava_callback - ERROR - Error processing callback for node 69c6f90e02375b4506dfc17c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33656 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:40:56,994 - lava_callback - INFO - Received callback for node 69c6f95e02375b4506dfc189 INFO: 172.20.0.1:33658 - "POST /node/69c6f95e02375b4506dfc189 HTTP/1.0" 202 Accepted 2026-03-29 21:40:56,999 - lava_callback - INFO - Starting async job submit for node 69c6f95e02375b4506dfc189 2026-03-29 21:40:57,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:40:57,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f95e02375b4506dfc189 HTTP/1.1" 200 None 2026-03-29 21:40:57,016 - lava_callback - DEBUG - Processing node 69c6f95e02375b4506dfc189: baseline-arm64-clabbe 2026-03-29 21:40:57,016 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f95e02375b4506dfc189 2026-03-29 21:40:57,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:40:57,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f95e02375b4506dfc189 HTTP/1.1" 200 None 2026-03-29 21:40:57,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:40:57,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:40:57,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:40:57,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f95e02375b4506dfc189 HTTP/1.1" 200 None 2026-03-29 21:40:57,076 - lava_callback - ERROR - Error processing callback for node 69c6f95e02375b4506dfc189: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50496 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:42:17,020 - lava_callback - INFO - Received callback for node 69c6f9af02375b4506dfc197 INFO: 172.20.0.1:50508 - "POST /node/69c6f9af02375b4506dfc197 HTTP/1.0" 202 Accepted 2026-03-29 21:42:17,026 - lava_callback - INFO - Starting async job submit for node 69c6f9af02375b4506dfc197 2026-03-29 21:42:17,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:42:17,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f9af02375b4506dfc197 HTTP/1.1" 200 None 2026-03-29 21:42:17,041 - lava_callback - DEBUG - Processing node 69c6f9af02375b4506dfc197: baseline-arm64-clabbe 2026-03-29 21:42:17,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f9af02375b4506dfc197 2026-03-29 21:42:17,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:42:17,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f9af02375b4506dfc197 HTTP/1.1" 200 None 2026-03-29 21:42:17,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:42:17,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:42:17,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:42:17,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f9af02375b4506dfc197 HTTP/1.1" 200 None 2026-03-29 21:42:17,124 - lava_callback - ERROR - Error processing callback for node 69c6f9af02375b4506dfc197: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56398 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:43:37,082 - lava_callback - INFO - Received callback for node 69c6f9f902375b4506dfc19f INFO: 172.20.0.1:56412 - "POST /node/69c6f9f902375b4506dfc19f HTTP/1.0" 202 Accepted 2026-03-29 21:43:37,088 - lava_callback - INFO - Starting async job submit for node 69c6f9f902375b4506dfc19f 2026-03-29 21:43:37,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:43:37,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f9f902375b4506dfc19f HTTP/1.1" 200 None 2026-03-29 21:43:37,104 - lava_callback - DEBUG - Processing node 69c6f9f902375b4506dfc19f: baseline-arm64-clabbe 2026-03-29 21:43:37,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6f9f902375b4506dfc19f 2026-03-29 21:43:37,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:43:37,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6f9f902375b4506dfc19f HTTP/1.1" 200 None 2026-03-29 21:43:37,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:43:37,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-29 21:43:37,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:43:37,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6f9f902375b4506dfc19f HTTP/1.1" 200 None 2026-03-29 21:43:37,165 - lava_callback - ERROR - Error processing callback for node 69c6f9f902375b4506dfc19f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60264 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:45:17,145 - lava_callback - INFO - Received callback for node 69c6fa5602375b4506dfc1ad INFO: 172.20.0.1:60278 - "POST /node/69c6fa5602375b4506dfc1ad HTTP/1.0" 202 Accepted 2026-03-29 21:45:17,151 - lava_callback - INFO - Starting async job submit for node 69c6fa5602375b4506dfc1ad 2026-03-29 21:45:17,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:45:17,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fa5602375b4506dfc1ad HTTP/1.1" 200 None 2026-03-29 21:45:17,167 - lava_callback - DEBUG - Processing node 69c6fa5602375b4506dfc1ad: baseline-arm64-clabbe 2026-03-29 21:45:17,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fa5602375b4506dfc1ad 2026-03-29 21:45:17,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:45:17,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fa5602375b4506dfc1ad HTTP/1.1" 200 None 2026-03-29 21:45:17,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:45:17,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:45:17,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:45:17,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fa5602375b4506dfc1ad HTTP/1.1" 200 None 2026-03-29 21:45:17,224 - lava_callback - ERROR - Error processing callback for node 69c6fa5602375b4506dfc1ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44888 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:46:13,301 - lava_callback - INFO - Received callback for node 69c6fa9e02375b4506dfc1b7 INFO: 172.20.0.1:44788 - "POST /node/69c6fa9e02375b4506dfc1b7 HTTP/1.0" 202 Accepted 2026-03-29 21:46:13,309 - lava_callback - INFO - Starting async job submit for node 69c6fa9e02375b4506dfc1b7 2026-03-29 21:46:13,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:46:13,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fa9e02375b4506dfc1b7 HTTP/1.1" 200 None 2026-03-29 21:46:13,327 - lava_callback - DEBUG - Processing node 69c6fa9e02375b4506dfc1b7: baseline-arm64-baylibre 2026-03-29 21:46:13,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fa9e02375b4506dfc1b7 2026-03-29 21:46:13,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:46:13,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fa9e02375b4506dfc1b7 HTTP/1.1" 200 None 2026-03-29 21:46:13,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:46:13,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:46:13,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:46:13,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fa9e02375b4506dfc1b7 HTTP/1.1" 200 None 2026-03-29 21:46:13,385 - lava_callback - ERROR - Error processing callback for node 69c6fa9e02375b4506dfc1b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57320 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:47:32,861 - lava_callback - INFO - Received callback for node 69c6faeb02375b4506dfc1c0 INFO: 172.20.0.1:54110 - "POST /node/69c6faeb02375b4506dfc1c0 HTTP/1.0" 202 Accepted 2026-03-29 21:47:32,867 - lava_callback - INFO - Starting async job submit for node 69c6faeb02375b4506dfc1c0 2026-03-29 21:47:32,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:47:32,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6faeb02375b4506dfc1c0 HTTP/1.1" 200 None 2026-03-29 21:47:32,885 - lava_callback - DEBUG - Processing node 69c6faeb02375b4506dfc1c0: baseline-arm64-baylibre 2026-03-29 21:47:32,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6faeb02375b4506dfc1c0 2026-03-29 21:47:32,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:47:32,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6faeb02375b4506dfc1c0 HTTP/1.1" 200 None 2026-03-29 21:47:32,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:47:32,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:47:32,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:47:32,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6faeb02375b4506dfc1c0 HTTP/1.1" 200 None 2026-03-29 21:47:32,944 - lava_callback - ERROR - Error processing callback for node 69c6faeb02375b4506dfc1c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39418 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:48:57,343 - lava_callback - INFO - Received callback for node 69c6fb3f02375b4506dfc1cf INFO: 172.20.0.1:39432 - "POST /node/69c6fb3f02375b4506dfc1cf HTTP/1.0" 202 Accepted 2026-03-29 21:48:57,349 - lava_callback - INFO - Starting async job submit for node 69c6fb3f02375b4506dfc1cf 2026-03-29 21:48:57,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:48:57,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fb3f02375b4506dfc1cf HTTP/1.1" 200 None 2026-03-29 21:48:57,364 - lava_callback - DEBUG - Processing node 69c6fb3f02375b4506dfc1cf: baseline-arm64-clabbe 2026-03-29 21:48:57,365 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fb3f02375b4506dfc1cf 2026-03-29 21:48:57,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:48:57,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fb3f02375b4506dfc1cf HTTP/1.1" 200 None 2026-03-29 21:48:57,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:48:57,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:48:57,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:48:57,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fb3f02375b4506dfc1cf HTTP/1.1" 200 None 2026-03-29 21:48:57,434 - lava_callback - ERROR - Error processing callback for node 69c6fb3f02375b4506dfc1cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57754 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:50:17,254 - lava_callback - INFO - Received callback for node 69c6fb8d02375b4506dfc1dc INFO: 172.20.0.1:57758 - "POST /node/69c6fb8d02375b4506dfc1dc HTTP/1.0" 202 Accepted 2026-03-29 21:50:17,260 - lava_callback - INFO - Starting async job submit for node 69c6fb8d02375b4506dfc1dc 2026-03-29 21:50:17,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:50:17,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fb8d02375b4506dfc1dc HTTP/1.1" 200 None 2026-03-29 21:50:17,276 - lava_callback - DEBUG - Processing node 69c6fb8d02375b4506dfc1dc: baseline-arm64-clabbe 2026-03-29 21:50:17,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fb8d02375b4506dfc1dc 2026-03-29 21:50:17,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:50:17,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fb8d02375b4506dfc1dc HTTP/1.1" 200 None 2026-03-29 21:50:17,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:50:17,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:50:17,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:50:17,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fb8d02375b4506dfc1dc HTTP/1.1" 200 None 2026-03-29 21:50:17,338 - lava_callback - ERROR - Error processing callback for node 69c6fb8d02375b4506dfc1dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54274 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:51:37,408 - lava_callback - INFO - Received callback for node 69c6fbde02375b4506dfc1e8 INFO: 172.20.0.1:54276 - "POST /node/69c6fbde02375b4506dfc1e8 HTTP/1.0" 202 Accepted 2026-03-29 21:51:37,413 - lava_callback - INFO - Starting async job submit for node 69c6fbde02375b4506dfc1e8 2026-03-29 21:51:37,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:51:37,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fbde02375b4506dfc1e8 HTTP/1.1" 200 None 2026-03-29 21:51:37,431 - lava_callback - DEBUG - Processing node 69c6fbde02375b4506dfc1e8: baseline-arm64-clabbe 2026-03-29 21:51:37,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fbde02375b4506dfc1e8 2026-03-29 21:51:37,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:51:37,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fbde02375b4506dfc1e8 HTTP/1.1" 200 None 2026-03-29 21:51:37,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:51:37,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:51:37,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:51:37,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fbde02375b4506dfc1e8 HTTP/1.1" 200 None 2026-03-29 21:51:37,485 - lava_callback - ERROR - Error processing callback for node 69c6fbde02375b4506dfc1e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34428 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:52:53,172 - lava_callback - INFO - Received callback for node 69c6fc2402375b4506dfc1f5 INFO: 172.20.0.1:54614 - "POST /node/69c6fc2402375b4506dfc1f5 HTTP/1.0" 202 Accepted 2026-03-29 21:52:53,177 - lava_callback - INFO - Starting async job submit for node 69c6fc2402375b4506dfc1f5 2026-03-29 21:52:53,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:52:53,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fc2402375b4506dfc1f5 HTTP/1.1" 200 None 2026-03-29 21:52:53,210 - lava_callback - DEBUG - Processing node 69c6fc2402375b4506dfc1f5: baseline-arm64-baylibre 2026-03-29 21:52:53,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fc2402375b4506dfc1f5 2026-03-29 21:52:53,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:52:53,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fc2402375b4506dfc1f5 HTTP/1.1" 200 None 2026-03-29 21:52:53,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:52:53,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:52:53,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:52:53,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fc2402375b4506dfc1f5 HTTP/1.1" 200 None 2026-03-29 21:52:53,273 - lava_callback - ERROR - Error processing callback for node 69c6fc2402375b4506dfc1f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:54:17,544 - lava_callback - INFO - Received callback for node 69c6fc7802375b4506dfc202 INFO: 172.20.0.1:41282 - "POST /node/69c6fc7802375b4506dfc202 HTTP/1.0" 202 Accepted 2026-03-29 21:54:17,549 - lava_callback - INFO - Starting async job submit for node 69c6fc7802375b4506dfc202 2026-03-29 21:54:17,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:54:17,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fc7802375b4506dfc202 HTTP/1.1" 200 None 2026-03-29 21:54:17,577 - lava_callback - DEBUG - Processing node 69c6fc7802375b4506dfc202: baseline-arm64-clabbe 2026-03-29 21:54:17,578 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fc7802375b4506dfc202 2026-03-29 21:54:17,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:54:17,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fc7802375b4506dfc202 HTTP/1.1" 200 None 2026-03-29 21:54:17,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:54:17,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:54:17,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:54:17,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fc7802375b4506dfc202 HTTP/1.1" 200 None 2026-03-29 21:54:17,684 - lava_callback - ERROR - Error processing callback for node 69c6fc7802375b4506dfc202: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:55:37,619 - lava_callback - INFO - Received callback for node 69c6fcc802375b4506dfc20d INFO: 172.20.0.1:47830 - "POST /node/69c6fcc802375b4506dfc20d HTTP/1.0" 202 Accepted 2026-03-29 21:55:37,625 - lava_callback - INFO - Starting async job submit for node 69c6fcc802375b4506dfc20d 2026-03-29 21:55:37,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:55:37,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fcc802375b4506dfc20d HTTP/1.1" 200 None 2026-03-29 21:55:37,642 - lava_callback - DEBUG - Processing node 69c6fcc802375b4506dfc20d: baseline-arm64-clabbe 2026-03-29 21:55:37,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fcc802375b4506dfc20d 2026-03-29 21:55:37,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:55:37,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fcc802375b4506dfc20d HTTP/1.1" 200 None 2026-03-29 21:55:37,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:55:37,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:55:37,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:55:37,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fcc802375b4506dfc20d HTTP/1.1" 200 None 2026-03-29 21:55:37,707 - lava_callback - ERROR - Error processing callback for node 69c6fcc802375b4506dfc20d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:56:57,680 - lava_callback - INFO - Received callback for node 69c6fd1902375b4506dfc21c INFO: 172.20.0.1:49748 - "POST /node/69c6fd1902375b4506dfc21c HTTP/1.0" 202 Accepted 2026-03-29 21:56:57,686 - lava_callback - INFO - Starting async job submit for node 69c6fd1902375b4506dfc21c 2026-03-29 21:56:57,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:56:57,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fd1902375b4506dfc21c HTTP/1.1" 200 None 2026-03-29 21:56:57,723 - lava_callback - DEBUG - Processing node 69c6fd1902375b4506dfc21c: baseline-arm64-clabbe 2026-03-29 21:56:57,723 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fd1902375b4506dfc21c 2026-03-29 21:56:57,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:56:57,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fd1902375b4506dfc21c HTTP/1.1" 200 None 2026-03-29 21:56:57,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:56:57,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:56:57,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:56:57,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fd1902375b4506dfc21c HTTP/1.1" 200 None 2026-03-29 21:56:57,805 - lava_callback - ERROR - Error processing callback for node 69c6fd1902375b4506dfc21c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54272 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:58:18,743 - lava_callback - INFO - Received callback for node 69c6fd6c02375b4506dfc229 INFO: 172.20.0.1:54282 - "POST /node/69c6fd6c02375b4506dfc229 HTTP/1.0" 202 Accepted 2026-03-29 21:58:18,749 - lava_callback - INFO - Starting async job submit for node 69c6fd6c02375b4506dfc229 2026-03-29 21:58:18,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:58:18,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fd6c02375b4506dfc229 HTTP/1.1" 200 None 2026-03-29 21:58:18,764 - lava_callback - DEBUG - Processing node 69c6fd6c02375b4506dfc229: baseline-arm64-clabbe 2026-03-29 21:58:18,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fd6c02375b4506dfc229 2026-03-29 21:58:18,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:58:18,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fd6c02375b4506dfc229 HTTP/1.1" 200 None 2026-03-29 21:58:18,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:58:18,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:58:18,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:58:18,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fd6c02375b4506dfc229 HTTP/1.1" 200 None 2026-03-29 21:58:18,822 - lava_callback - ERROR - Error processing callback for node 69c6fd6c02375b4506dfc229: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33598 - "GET / HTTP/1.0" 200 OK 2026-03-29 21:59:37,761 - lava_callback - INFO - Received callback for node 69c6fdc102375b4506dfc233 INFO: 172.20.0.1:33604 - "POST /node/69c6fdc102375b4506dfc233 HTTP/1.0" 202 Accepted 2026-03-29 21:59:37,767 - lava_callback - INFO - Starting async job submit for node 69c6fdc102375b4506dfc233 2026-03-29 21:59:37,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:59:37,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fdc102375b4506dfc233 HTTP/1.1" 200 None 2026-03-29 21:59:37,782 - lava_callback - DEBUG - Processing node 69c6fdc102375b4506dfc233: baseline-arm64-clabbe 2026-03-29 21:59:37,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fdc102375b4506dfc233 2026-03-29 21:59:37,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:59:37,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fdc102375b4506dfc233 HTTP/1.1" 200 None 2026-03-29 21:59:37,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:59:37,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 21:59:37,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 21:59:37,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fdc102375b4506dfc233 HTTP/1.1" 200 None 2026-03-29 21:59:37,837 - lava_callback - ERROR - Error processing callback for node 69c6fdc102375b4506dfc233: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49954 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:01:17,833 - lava_callback - INFO - Received callback for node 69c6fe1702375b4506dfc23f INFO: 172.20.0.1:49968 - "POST /node/69c6fe1702375b4506dfc23f HTTP/1.0" 202 Accepted 2026-03-29 22:01:17,838 - lava_callback - INFO - Starting async job submit for node 69c6fe1702375b4506dfc23f 2026-03-29 22:01:17,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:01:17,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fe1702375b4506dfc23f HTTP/1.1" 200 None 2026-03-29 22:01:17,855 - lava_callback - DEBUG - Processing node 69c6fe1702375b4506dfc23f: baseline-arm64-clabbe 2026-03-29 22:01:17,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fe1702375b4506dfc23f 2026-03-29 22:01:17,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:01:17,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fe1702375b4506dfc23f HTTP/1.1" 200 None 2026-03-29 22:01:17,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:01:17,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 22:01:17,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:01:17,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fe1702375b4506dfc23f HTTP/1.1" 200 None 2026-03-29 22:01:17,910 - lava_callback - ERROR - Error processing callback for node 69c6fe1702375b4506dfc23f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47024 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:02:37,932 - lava_callback - INFO - Received callback for node 69c6fe6b02375b4506dfc24d INFO: 172.20.0.1:47038 - "POST /node/69c6fe6b02375b4506dfc24d HTTP/1.0" 202 Accepted 2026-03-29 22:02:37,938 - lava_callback - INFO - Starting async job submit for node 69c6fe6b02375b4506dfc24d 2026-03-29 22:02:37,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:02:37,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fe6b02375b4506dfc24d HTTP/1.1" 200 None 2026-03-29 22:02:37,954 - lava_callback - DEBUG - Processing node 69c6fe6b02375b4506dfc24d: baseline-arm64-clabbe 2026-03-29 22:02:37,954 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6fe6b02375b4506dfc24d 2026-03-29 22:02:37,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:02:37,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6fe6b02375b4506dfc24d HTTP/1.1" 200 None 2026-03-29 22:02:37,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:02:37,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 22:02:37,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:02:38,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6fe6b02375b4506dfc24d HTTP/1.1" 200 None 2026-03-29 22:02:38,012 - lava_callback - ERROR - Error processing callback for node 69c6fe6b02375b4506dfc24d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:03:57,935 - lava_callback - INFO - Received callback for node 69c6febf02375b4506dfc25a INFO: 172.20.0.1:43274 - "POST /node/69c6febf02375b4506dfc25a HTTP/1.0" 202 Accepted 2026-03-29 22:03:57,940 - lava_callback - INFO - Starting async job submit for node 69c6febf02375b4506dfc25a 2026-03-29 22:03:57,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:03:57,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6febf02375b4506dfc25a HTTP/1.1" 200 None 2026-03-29 22:03:57,957 - lava_callback - DEBUG - Processing node 69c6febf02375b4506dfc25a: baseline-arm64-clabbe 2026-03-29 22:03:57,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6febf02375b4506dfc25a 2026-03-29 22:03:57,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:03:57,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6febf02375b4506dfc25a HTTP/1.1" 200 None 2026-03-29 22:03:57,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:03:57,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 22:03:57,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:03:58,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6febf02375b4506dfc25a HTTP/1.1" 200 None 2026-03-29 22:03:58,012 - lava_callback - ERROR - Error processing callback for node 69c6febf02375b4506dfc25a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54770 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:05:17,982 - lava_callback - INFO - Received callback for node 69c6ff1302375b4506dfc269 INFO: 172.20.0.1:54776 - "POST /node/69c6ff1302375b4506dfc269 HTTP/1.0" 202 Accepted 2026-03-29 22:05:17,988 - lava_callback - INFO - Starting async job submit for node 69c6ff1302375b4506dfc269 2026-03-29 22:05:17,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:05:18,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ff1302375b4506dfc269 HTTP/1.1" 200 None 2026-03-29 22:05:18,004 - lava_callback - DEBUG - Processing node 69c6ff1302375b4506dfc269: baseline-arm64-clabbe 2026-03-29 22:05:18,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6ff1302375b4506dfc269 2026-03-29 22:05:18,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:05:18,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ff1302375b4506dfc269 HTTP/1.1" 200 None 2026-03-29 22:05:18,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:05:18,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-29 22:05:18,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:05:18,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6ff1302375b4506dfc269 HTTP/1.1" 200 None 2026-03-29 22:05:18,064 - lava_callback - ERROR - Error processing callback for node 69c6ff1302375b4506dfc269: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44214 - "GET /node/699397c9cebe99dd8e074a26%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48020 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:40:35,924 - lava_callback - INFO - Received callback for node 69c7075702375b4506dfc2aa INFO: 172.20.0.1:43138 - "POST /node/69c7075702375b4506dfc2aa HTTP/1.0" 202 Accepted 2026-03-29 22:40:35,930 - lava_callback - INFO - Starting async job submit for node 69c7075702375b4506dfc2aa 2026-03-29 22:40:35,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:40:35,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7075702375b4506dfc2aa HTTP/1.1" 200 None 2026-03-29 22:40:35,947 - lava_callback - DEBUG - Processing node 69c7075702375b4506dfc2aa: baseline-arm-baylibre 2026-03-29 22:40:35,947 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7075702375b4506dfc2aa 2026-03-29 22:40:35,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:40:35,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7075702375b4506dfc2aa HTTP/1.1" 200 None 2026-03-29 22:40:35,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:40:35,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:40:35,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:40:36,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7075702375b4506dfc2aa HTTP/1.1" 200 None 2026-03-29 22:40:36,009 - lava_callback - ERROR - Error processing callback for node 69c7075702375b4506dfc2aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47446 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:41:15,678 - lava_callback - INFO - Received callback for node 69c7077d02375b4506dfc2b8 INFO: 172.20.0.1:47452 - "POST /node/69c7077d02375b4506dfc2b8 HTTP/1.0" 202 Accepted 2026-03-29 22:41:15,684 - lava_callback - INFO - Starting async job submit for node 69c7077d02375b4506dfc2b8 2026-03-29 22:41:15,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:15,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7077d02375b4506dfc2b8 HTTP/1.1" 200 None 2026-03-29 22:41:15,700 - lava_callback - DEBUG - Processing node 69c7077d02375b4506dfc2b8: baseline-arm-baylibre 2026-03-29 22:41:15,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7077d02375b4506dfc2b8 2026-03-29 22:41:15,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:15,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7077d02375b4506dfc2b8 HTTP/1.1" 200 None 2026-03-29 22:41:15,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:15,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:41:15,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:15,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7077d02375b4506dfc2b8 HTTP/1.1" 200 None 2026-03-29 22:41:15,820 - lava_callback - ERROR - Error processing callback for node 69c7077d02375b4506dfc2b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55292 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:41:55,755 - lava_callback - INFO - Received callback for node 69c707a102375b4506dfc2c3 INFO: 172.20.0.1:37728 - "POST /node/69c707a102375b4506dfc2c3 HTTP/1.0" 202 Accepted 2026-03-29 22:41:55,760 - lava_callback - INFO - Starting async job submit for node 69c707a102375b4506dfc2c3 2026-03-29 22:41:55,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:55,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c707a102375b4506dfc2c3 HTTP/1.1" 200 None 2026-03-29 22:41:55,780 - lava_callback - DEBUG - Processing node 69c707a102375b4506dfc2c3: baseline-arm-baylibre 2026-03-29 22:41:55,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c707a102375b4506dfc2c3 2026-03-29 22:41:55,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:55,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c707a102375b4506dfc2c3 HTTP/1.1" 200 None 2026-03-29 22:41:55,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:55,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:41:55,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:41:55,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c707a102375b4506dfc2c3 HTTP/1.1" 200 None 2026-03-29 22:41:55,842 - lava_callback - ERROR - Error processing callback for node 69c707a102375b4506dfc2c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37614 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:42:15,737 - lava_callback - INFO - Received callback for node 69c707c502375b4506dfc2cf INFO: 172.20.0.1:37630 - "POST /node/69c707c502375b4506dfc2cf HTTP/1.0" 202 Accepted 2026-03-29 22:42:15,742 - lava_callback - INFO - Starting async job submit for node 69c707c502375b4506dfc2cf 2026-03-29 22:42:15,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:15,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c707c502375b4506dfc2cf HTTP/1.1" 200 None 2026-03-29 22:42:15,772 - lava_callback - DEBUG - Processing node 69c707c502375b4506dfc2cf: baseline-arm-baylibre 2026-03-29 22:42:15,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c707c502375b4506dfc2cf 2026-03-29 22:42:15,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:15,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c707c502375b4506dfc2cf HTTP/1.1" 200 None 2026-03-29 22:42:15,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:15,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:42:15,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:15,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c707c502375b4506dfc2cf HTTP/1.1" 200 None 2026-03-29 22:42:15,840 - lava_callback - ERROR - Error processing callback for node 69c707c502375b4506dfc2cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40934 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:42:55,765 - lava_callback - INFO - Received callback for node 69c707e902375b4506dfc2da INFO: 172.20.0.1:46488 - "POST /node/69c707e902375b4506dfc2da HTTP/1.0" 202 Accepted 2026-03-29 22:42:55,771 - lava_callback - INFO - Starting async job submit for node 69c707e902375b4506dfc2da 2026-03-29 22:42:55,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:55,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c707e902375b4506dfc2da HTTP/1.1" 200 None 2026-03-29 22:42:55,787 - lava_callback - DEBUG - Processing node 69c707e902375b4506dfc2da: baseline-arm-baylibre 2026-03-29 22:42:55,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c707e902375b4506dfc2da 2026-03-29 22:42:55,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:55,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c707e902375b4506dfc2da HTTP/1.1" 200 None 2026-03-29 22:42:55,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:55,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:42:55,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:42:55,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c707e902375b4506dfc2da HTTP/1.1" 200 None 2026-03-29 22:42:55,841 - lava_callback - ERROR - Error processing callback for node 69c707e902375b4506dfc2da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53390 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:43:35,870 - lava_callback - INFO - Received callback for node 69c7080e02375b4506dfc2e5 INFO: 172.20.0.1:53400 - "POST /node/69c7080e02375b4506dfc2e5 HTTP/1.0" 202 Accepted 2026-03-29 22:43:35,876 - lava_callback - INFO - Starting async job submit for node 69c7080e02375b4506dfc2e5 2026-03-29 22:43:35,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:43:35,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7080e02375b4506dfc2e5 HTTP/1.1" 200 None 2026-03-29 22:43:35,907 - lava_callback - DEBUG - Processing node 69c7080e02375b4506dfc2e5: baseline-arm-baylibre 2026-03-29 22:43:35,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7080e02375b4506dfc2e5 2026-03-29 22:43:35,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:43:35,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7080e02375b4506dfc2e5 HTTP/1.1" 200 None 2026-03-29 22:43:35,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:43:35,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:43:35,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:43:35,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7080e02375b4506dfc2e5 HTTP/1.1" 200 None 2026-03-29 22:43:35,972 - lava_callback - ERROR - Error processing callback for node 69c7080e02375b4506dfc2e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60258 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:44:15,838 - lava_callback - INFO - Received callback for node 69c7083302375b4506dfc2f7 INFO: 172.20.0.1:60270 - "POST /node/69c7083302375b4506dfc2f7 HTTP/1.0" 202 Accepted 2026-03-29 22:44:15,844 - lava_callback - INFO - Starting async job submit for node 69c7083302375b4506dfc2f7 2026-03-29 22:44:15,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:15,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7083302375b4506dfc2f7 HTTP/1.1" 200 None 2026-03-29 22:44:15,860 - lava_callback - DEBUG - Processing node 69c7083302375b4506dfc2f7: baseline-arm-baylibre 2026-03-29 22:44:15,860 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7083302375b4506dfc2f7 2026-03-29 22:44:15,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:15,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7083302375b4506dfc2f7 HTTP/1.1" 200 None 2026-03-29 22:44:15,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:15,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:44:15,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:15,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7083302375b4506dfc2f7 HTTP/1.1" 200 None 2026-03-29 22:44:15,921 - lava_callback - ERROR - Error processing callback for node 69c7083302375b4506dfc2f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:44:55,914 - lava_callback - INFO - Received callback for node 69c7085902375b4506dfc303 INFO: 172.20.0.1:54404 - "POST /node/69c7085902375b4506dfc303 HTTP/1.0" 202 Accepted 2026-03-29 22:44:55,919 - lava_callback - INFO - Starting async job submit for node 69c7085902375b4506dfc303 2026-03-29 22:44:55,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:55,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7085902375b4506dfc303 HTTP/1.1" 200 None 2026-03-29 22:44:55,936 - lava_callback - DEBUG - Processing node 69c7085902375b4506dfc303: baseline-arm-baylibre 2026-03-29 22:44:55,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7085902375b4506dfc303 2026-03-29 22:44:55,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:55,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7085902375b4506dfc303 HTTP/1.1" 200 None 2026-03-29 22:44:55,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:55,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:44:55,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:44:55,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7085902375b4506dfc303 HTTP/1.1" 200 None 2026-03-29 22:44:55,997 - lava_callback - ERROR - Error processing callback for node 69c7085902375b4506dfc303: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39132 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:45:35,911 - lava_callback - INFO - Received callback for node 69c7087e02375b4506dfc311 INFO: 172.20.0.1:52768 - "POST /node/69c7087e02375b4506dfc311 HTTP/1.0" 202 Accepted 2026-03-29 22:45:35,917 - lava_callback - INFO - Starting async job submit for node 69c7087e02375b4506dfc311 2026-03-29 22:45:35,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:45:35,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7087e02375b4506dfc311 HTTP/1.1" 200 None 2026-03-29 22:45:35,933 - lava_callback - DEBUG - Processing node 69c7087e02375b4506dfc311: baseline-arm-baylibre 2026-03-29 22:45:35,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7087e02375b4506dfc311 2026-03-29 22:45:35,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:45:35,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7087e02375b4506dfc311 HTTP/1.1" 200 None 2026-03-29 22:45:35,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:45:35,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:45:35,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:45:35,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7087e02375b4506dfc311 HTTP/1.1" 200 None 2026-03-29 22:45:35,996 - lava_callback - ERROR - Error processing callback for node 69c7087e02375b4506dfc311: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41998 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:46:16,037 - lava_callback - INFO - Received callback for node 69c708a302375b4506dfc320 INFO: 172.20.0.1:42002 - "POST /node/69c708a302375b4506dfc320 HTTP/1.0" 202 Accepted 2026-03-29 22:46:16,042 - lava_callback - INFO - Starting async job submit for node 69c708a302375b4506dfc320 2026-03-29 22:46:16,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:16,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c708a302375b4506dfc320 HTTP/1.1" 200 None 2026-03-29 22:46:16,060 - lava_callback - DEBUG - Processing node 69c708a302375b4506dfc320: baseline-arm-baylibre 2026-03-29 22:46:16,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c708a302375b4506dfc320 2026-03-29 22:46:16,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:16,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c708a302375b4506dfc320 HTTP/1.1" 200 None 2026-03-29 22:46:16,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:16,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:46:16,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:16,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c708a302375b4506dfc320 HTTP/1.1" 200 None 2026-03-29 22:46:16,134 - lava_callback - ERROR - Error processing callback for node 69c708a302375b4506dfc320: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35766 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:46:55,989 - lava_callback - INFO - Received callback for node 69c708cb02375b4506dfc32d INFO: 172.20.0.1:44378 - "POST /node/69c708cb02375b4506dfc32d HTTP/1.0" 202 Accepted 2026-03-29 22:46:55,995 - lava_callback - INFO - Starting async job submit for node 69c708cb02375b4506dfc32d 2026-03-29 22:46:55,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:56,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c708cb02375b4506dfc32d HTTP/1.1" 200 None 2026-03-29 22:46:56,021 - lava_callback - DEBUG - Processing node 69c708cb02375b4506dfc32d: baseline-arm-baylibre 2026-03-29 22:46:56,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c708cb02375b4506dfc32d 2026-03-29 22:46:56,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:56,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c708cb02375b4506dfc32d HTTP/1.1" 200 None 2026-03-29 22:46:56,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:56,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:46:56,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:46:56,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c708cb02375b4506dfc32d HTTP/1.1" 200 None 2026-03-29 22:46:56,074 - lava_callback - ERROR - Error processing callback for node 69c708cb02375b4506dfc32d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52526 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:47:16,008 - lava_callback - INFO - Received callback for node 69c708f102375b4506dfc338 INFO: 172.20.0.1:52530 - "POST /node/69c708f102375b4506dfc338 HTTP/1.0" 202 Accepted 2026-03-29 22:47:16,014 - lava_callback - INFO - Starting async job submit for node 69c708f102375b4506dfc338 2026-03-29 22:47:16,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:47:16,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c708f102375b4506dfc338 HTTP/1.1" 200 None 2026-03-29 22:47:16,031 - lava_callback - DEBUG - Processing node 69c708f102375b4506dfc338: baseline-arm-baylibre 2026-03-29 22:47:16,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c708f102375b4506dfc338 2026-03-29 22:47:16,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:47:16,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c708f102375b4506dfc338 HTTP/1.1" 200 None 2026-03-29 22:47:16,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:47:16,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:47:16,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:47:16,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c708f102375b4506dfc338 HTTP/1.1" 200 None 2026-03-29 22:47:16,090 - lava_callback - ERROR - Error processing callback for node 69c708f102375b4506dfc338: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38992 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:48:16,107 - lava_callback - INFO - Received callback for node 69c7091b02375b4506dfc344 INFO: 172.20.0.1:38998 - "POST /node/69c7091b02375b4506dfc344 HTTP/1.0" 202 Accepted 2026-03-29 22:48:16,113 - lava_callback - INFO - Starting async job submit for node 69c7091b02375b4506dfc344 2026-03-29 22:48:16,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:16,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7091b02375b4506dfc344 HTTP/1.1" 200 None 2026-03-29 22:48:16,129 - lava_callback - DEBUG - Processing node 69c7091b02375b4506dfc344: baseline-arm-baylibre 2026-03-29 22:48:16,130 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7091b02375b4506dfc344 2026-03-29 22:48:16,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:16,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7091b02375b4506dfc344 HTTP/1.1" 200 None 2026-03-29 22:48:16,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:16,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:48:16,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:16,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7091b02375b4506dfc344 HTTP/1.1" 200 None 2026-03-29 22:48:16,207 - lava_callback - ERROR - Error processing callback for node 69c7091b02375b4506dfc344: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38182 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:48:56,094 - lava_callback - INFO - Received callback for node 69c7094302375b4506dfc351 INFO: 172.20.0.1:34740 - "POST /node/69c7094302375b4506dfc351 HTTP/1.0" 202 Accepted 2026-03-29 22:48:56,099 - lava_callback - INFO - Starting async job submit for node 69c7094302375b4506dfc351 2026-03-29 22:48:56,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:56,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7094302375b4506dfc351 HTTP/1.1" 200 None 2026-03-29 22:48:56,116 - lava_callback - DEBUG - Processing node 69c7094302375b4506dfc351: baseline-arm-baylibre 2026-03-29 22:48:56,116 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7094302375b4506dfc351 2026-03-29 22:48:56,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:56,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7094302375b4506dfc351 HTTP/1.1" 200 None 2026-03-29 22:48:56,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:56,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:48:56,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:48:56,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7094302375b4506dfc351 HTTP/1.1" 200 None 2026-03-29 22:48:56,173 - lava_callback - ERROR - Error processing callback for node 69c7094302375b4506dfc351: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50728 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:49:36,136 - lava_callback - INFO - Received callback for node 69c7096d02375b4506dfc35e INFO: 172.20.0.1:56658 - "POST /node/69c7096d02375b4506dfc35e HTTP/1.0" 202 Accepted 2026-03-29 22:49:36,142 - lava_callback - INFO - Starting async job submit for node 69c7096d02375b4506dfc35e 2026-03-29 22:49:36,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:49:36,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7096d02375b4506dfc35e HTTP/1.1" 200 None 2026-03-29 22:49:36,158 - lava_callback - DEBUG - Processing node 69c7096d02375b4506dfc35e: baseline-arm-baylibre 2026-03-29 22:49:36,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7096d02375b4506dfc35e 2026-03-29 22:49:36,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:49:36,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7096d02375b4506dfc35e HTTP/1.1" 200 None 2026-03-29 22:49:36,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:49:36,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:49:36,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:49:36,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7096d02375b4506dfc35e HTTP/1.1" 200 None 2026-03-29 22:49:36,220 - lava_callback - ERROR - Error processing callback for node 69c7096d02375b4506dfc35e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39706 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:50:16,150 - lava_callback - INFO - Received callback for node 69c7099602375b4506dfc36d INFO: 172.20.0.1:39714 - "POST /node/69c7099602375b4506dfc36d HTTP/1.0" 202 Accepted 2026-03-29 22:50:16,156 - lava_callback - INFO - Starting async job submit for node 69c7099602375b4506dfc36d 2026-03-29 22:50:16,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:16,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7099602375b4506dfc36d HTTP/1.1" 200 None 2026-03-29 22:50:16,172 - lava_callback - DEBUG - Processing node 69c7099602375b4506dfc36d: baseline-arm-baylibre 2026-03-29 22:50:16,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7099602375b4506dfc36d 2026-03-29 22:50:16,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:16,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7099602375b4506dfc36d HTTP/1.1" 200 None 2026-03-29 22:50:16,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:16,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:50:16,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:16,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7099602375b4506dfc36d HTTP/1.1" 200 None 2026-03-29 22:50:16,256 - lava_callback - ERROR - Error processing callback for node 69c7099602375b4506dfc36d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46682 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:50:56,232 - lava_callback - INFO - Received callback for node 69c709bd02375b4506dfc37e INFO: 172.20.0.1:53574 - "POST /node/69c709bd02375b4506dfc37e HTTP/1.0" 202 Accepted 2026-03-29 22:50:56,237 - lava_callback - INFO - Starting async job submit for node 69c709bd02375b4506dfc37e 2026-03-29 22:50:56,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:56,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c709bd02375b4506dfc37e HTTP/1.1" 200 None 2026-03-29 22:50:56,255 - lava_callback - DEBUG - Processing node 69c709bd02375b4506dfc37e: baseline-arm-baylibre 2026-03-29 22:50:56,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c709bd02375b4506dfc37e 2026-03-29 22:50:56,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:56,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c709bd02375b4506dfc37e HTTP/1.1" 200 None 2026-03-29 22:50:56,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:56,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:50:56,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:50:56,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c709bd02375b4506dfc37e HTTP/1.1" 200 None 2026-03-29 22:50:56,320 - lava_callback - ERROR - Error processing callback for node 69c709bd02375b4506dfc37e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:51:36,287 - lava_callback - INFO - Received callback for node 69c709e402375b4506dfc38b INFO: 172.20.0.1:40356 - "POST /node/69c709e402375b4506dfc38b HTTP/1.0" 202 Accepted 2026-03-29 22:51:36,293 - lava_callback - INFO - Starting async job submit for node 69c709e402375b4506dfc38b 2026-03-29 22:51:36,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:36,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c709e402375b4506dfc38b HTTP/1.1" 200 None 2026-03-29 22:51:36,309 - lava_callback - DEBUG - Processing node 69c709e402375b4506dfc38b: baseline-arm-baylibre 2026-03-29 22:51:36,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c709e402375b4506dfc38b 2026-03-29 22:51:36,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:36,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c709e402375b4506dfc38b HTTP/1.1" 200 None 2026-03-29 22:51:36,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:36,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:51:36,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:36,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c709e402375b4506dfc38b HTTP/1.1" 200 None 2026-03-29 22:51:36,370 - lava_callback - ERROR - Error processing callback for node 69c709e402375b4506dfc38b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46296 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:51:56,234 - lava_callback - INFO - Received callback for node 69c70a0a02375b4506dfc39c INFO: 172.20.0.1:49936 - "POST /node/69c70a0a02375b4506dfc39c HTTP/1.0" 202 Accepted 2026-03-29 22:51:56,240 - lava_callback - INFO - Starting async job submit for node 69c70a0a02375b4506dfc39c 2026-03-29 22:51:56,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:56,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a0a02375b4506dfc39c HTTP/1.1" 200 None 2026-03-29 22:51:56,256 - lava_callback - DEBUG - Processing node 69c70a0a02375b4506dfc39c: baseline-arm-baylibre 2026-03-29 22:51:56,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70a0a02375b4506dfc39c 2026-03-29 22:51:56,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:56,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a0a02375b4506dfc39c HTTP/1.1" 200 None 2026-03-29 22:51:56,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:56,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:51:56,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:51:56,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70a0a02375b4506dfc39c HTTP/1.1" 200 None 2026-03-29 22:51:56,320 - lava_callback - ERROR - Error processing callback for node 69c70a0a02375b4506dfc39c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET /node/69831396409235c29c0def31%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:52:40,042 - lava_callback - INFO - Received callback for node 69c70a3202375b4506dfc3af INFO: 172.20.0.1:35830 - "POST /node/69c70a3202375b4506dfc3af HTTP/1.0" 202 Accepted 2026-03-29 22:52:40,048 - lava_callback - INFO - Starting async job submit for node 69c70a3202375b4506dfc3af 2026-03-29 22:52:40,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:52:40,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a3202375b4506dfc3af HTTP/1.1" 200 None 2026-03-29 22:52:40,068 - lava_callback - DEBUG - Processing node 69c70a3202375b4506dfc3af: baseline-arm-clabbe 2026-03-29 22:52:40,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70a3202375b4506dfc3af 2026-03-29 22:52:40,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:52:40,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a3202375b4506dfc3af HTTP/1.1" 200 None 2026-03-29 22:52:40,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:52:40,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:52:40,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:52:40,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70a3202375b4506dfc3af HTTP/1.1" 200 None 2026-03-29 22:52:40,140 - lava_callback - ERROR - Error processing callback for node 69c70a3202375b4506dfc3af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52880 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 22:53:16,360 - lava_callback - INFO - Received callback for node 69c70a5702375b4506dfc3bb INFO: 172.20.0.1:52894 - "POST /node/69c70a5702375b4506dfc3bb HTTP/1.0" 202 Accepted 2026-03-29 22:53:16,365 - lava_callback - INFO - Starting async job submit for node 69c70a5702375b4506dfc3bb 2026-03-29 22:53:16,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:53:16,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a5702375b4506dfc3bb HTTP/1.1" 200 None 2026-03-29 22:53:16,381 - lava_callback - DEBUG - Processing node 69c70a5702375b4506dfc3bb: baseline-arm-baylibre 2026-03-29 22:53:16,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70a5702375b4506dfc3bb 2026-03-29 22:53:16,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:53:16,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a5702375b4506dfc3bb HTTP/1.1" 200 None 2026-03-29 22:53:16,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:53:16,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:53:16,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:53:16,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70a5702375b4506dfc3bb HTTP/1.1" 200 None 2026-03-29 22:53:16,449 - lava_callback - ERROR - Error processing callback for node 69c70a5702375b4506dfc3bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59714 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:54:00,250 - lava_callback - INFO - Received callback for node 69c70a8102375b4506dfc3e0 INFO: 172.20.0.1:37168 - "POST /node/69c70a8102375b4506dfc3e0 HTTP/1.0" 202 Accepted 2026-03-29 22:54:00,255 - lava_callback - INFO - Starting async job submit for node 69c70a8102375b4506dfc3e0 2026-03-29 22:54:00,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:00,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a8102375b4506dfc3e0 HTTP/1.1" 200 None 2026-03-29 22:54:00,271 - lava_callback - DEBUG - Processing node 69c70a8102375b4506dfc3e0: baseline-arm-clabbe 2026-03-29 22:54:00,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70a8102375b4506dfc3e0 2026-03-29 22:54:00,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:00,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70a8102375b4506dfc3e0 HTTP/1.1" 200 None 2026-03-29 22:54:00,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:00,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:54:00,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:00,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70a8102375b4506dfc3e0 HTTP/1.1" 200 None 2026-03-29 22:54:00,346 - lava_callback - ERROR - Error processing callback for node 69c70a8102375b4506dfc3e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55104 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:54:36,433 - lava_callback - INFO - Received callback for node 69c70aa502375b4506dfc3f9 INFO: 172.20.0.1:58702 - "POST /node/69c70aa502375b4506dfc3f9 HTTP/1.0" 202 Accepted 2026-03-29 22:54:36,438 - lava_callback - INFO - Starting async job submit for node 69c70aa502375b4506dfc3f9 2026-03-29 22:54:36,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:36,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70aa502375b4506dfc3f9 HTTP/1.1" 200 None 2026-03-29 22:54:36,454 - lava_callback - DEBUG - Processing node 69c70aa502375b4506dfc3f9: baseline-arm-baylibre 2026-03-29 22:54:36,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70aa502375b4506dfc3f9 2026-03-29 22:54:36,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:36,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70aa502375b4506dfc3f9 HTTP/1.1" 200 None 2026-03-29 22:54:36,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:36,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:54:36,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:54:36,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70aa502375b4506dfc3f9 HTTP/1.1" 200 None 2026-03-29 22:54:36,508 - lava_callback - ERROR - Error processing callback for node 69c70aa502375b4506dfc3f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48076 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:55:20,062 - lava_callback - INFO - Received callback for node 69c70ace02375b4506dfc41c INFO: 172.20.0.1:52762 - "POST /node/69c70ace02375b4506dfc41c HTTP/1.0" 202 Accepted 2026-03-29 22:55:20,068 - lava_callback - INFO - Starting async job submit for node 69c70ace02375b4506dfc41c 2026-03-29 22:55:20,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:55:20,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70ace02375b4506dfc41c HTTP/1.1" 200 None 2026-03-29 22:55:20,086 - lava_callback - DEBUG - Processing node 69c70ace02375b4506dfc41c: baseline-arm-clabbe 2026-03-29 22:55:20,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70ace02375b4506dfc41c 2026-03-29 22:55:20,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:55:20,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70ace02375b4506dfc41c HTTP/1.1" 200 None 2026-03-29 22:55:20,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:55:20,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:55:20,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:55:20,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70ace02375b4506dfc41c HTTP/1.1" 200 None 2026-03-29 22:55:20,144 - lava_callback - ERROR - Error processing callback for node 69c70ace02375b4506dfc41c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52958 - "GET /node/698f86863013abef95d0215f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53258 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:56:00,102 - lava_callback - INFO - Received callback for node 69c70af502375b4506dfc42b INFO: 172.20.0.1:50012 - "POST /node/69c70af502375b4506dfc42b HTTP/1.0" 202 Accepted 2026-03-29 22:56:00,108 - lava_callback - INFO - Starting async job submit for node 69c70af502375b4506dfc42b 2026-03-29 22:56:00,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:00,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70af502375b4506dfc42b HTTP/1.1" 200 None 2026-03-29 22:56:00,125 - lava_callback - DEBUG - Processing node 69c70af502375b4506dfc42b: baseline-arm-clabbe 2026-03-29 22:56:00,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70af502375b4506dfc42b 2026-03-29 22:56:00,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:00,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70af502375b4506dfc42b HTTP/1.1" 200 None 2026-03-29 22:56:00,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:00,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:56:00,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:00,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70af502375b4506dfc42b HTTP/1.1" 200 None 2026-03-29 22:56:00,228 - lava_callback - ERROR - Error processing callback for node 69c70af502375b4506dfc42b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36996 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:56:40,187 - lava_callback - INFO - Received callback for node 69c70b1b02375b4506dfc438 INFO: 172.20.0.1:45116 - "POST /node/69c70b1b02375b4506dfc438 HTTP/1.0" 202 Accepted 2026-03-29 22:56:40,192 - lava_callback - INFO - Starting async job submit for node 69c70b1b02375b4506dfc438 2026-03-29 22:56:40,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:40,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70b1b02375b4506dfc438 HTTP/1.1" 200 None 2026-03-29 22:56:40,209 - lava_callback - DEBUG - Processing node 69c70b1b02375b4506dfc438: baseline-arm-clabbe 2026-03-29 22:56:40,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70b1b02375b4506dfc438 2026-03-29 22:56:40,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:40,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70b1b02375b4506dfc438 HTTP/1.1" 200 None 2026-03-29 22:56:40,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:40,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:56:40,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:56:40,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70b1b02375b4506dfc438 HTTP/1.1" 200 None 2026-03-29 22:56:40,270 - lava_callback - ERROR - Error processing callback for node 69c70b1b02375b4506dfc438: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41952 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:58:00,271 - lava_callback - INFO - Received callback for node 69c70b6602375b4506dfc44e INFO: 172.20.0.1:40212 - "POST /node/69c70b6602375b4506dfc44e HTTP/1.0" 202 Accepted 2026-03-29 22:58:00,277 - lava_callback - INFO - Starting async job submit for node 69c70b6602375b4506dfc44e 2026-03-29 22:58:00,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:00,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70b6602375b4506dfc44e HTTP/1.1" 200 None 2026-03-29 22:58:00,300 - lava_callback - DEBUG - Processing node 69c70b6602375b4506dfc44e: baseline-arm-clabbe 2026-03-29 22:58:00,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70b6602375b4506dfc44e 2026-03-29 22:58:00,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:00,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70b6602375b4506dfc44e HTTP/1.1" 200 None 2026-03-29 22:58:00,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:00,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:58:00,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:00,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70b6602375b4506dfc44e HTTP/1.1" 200 None 2026-03-29 22:58:00,364 - lava_callback - ERROR - Error processing callback for node 69c70b6602375b4506dfc44e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45720 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:58:40,197 - lava_callback - INFO - Received callback for node 69c70b8c02375b4506dfc45d INFO: 172.20.0.1:54080 - "POST /node/69c70b8c02375b4506dfc45d HTTP/1.0" 202 Accepted 2026-03-29 22:58:40,203 - lava_callback - INFO - Starting async job submit for node 69c70b8c02375b4506dfc45d 2026-03-29 22:58:40,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:40,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70b8c02375b4506dfc45d HTTP/1.1" 200 None 2026-03-29 22:58:40,219 - lava_callback - DEBUG - Processing node 69c70b8c02375b4506dfc45d: baseline-arm-clabbe 2026-03-29 22:58:40,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70b8c02375b4506dfc45d 2026-03-29 22:58:40,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:40,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70b8c02375b4506dfc45d HTTP/1.1" 200 None 2026-03-29 22:58:40,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:40,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 22:58:40,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:58:40,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70b8c02375b4506dfc45d HTTP/1.1" 200 None 2026-03-29 22:58:40,316 - lava_callback - ERROR - Error processing callback for node 69c70b8c02375b4506dfc45d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32796 - "GET / HTTP/1.0" 200 OK 2026-03-29 22:59:20,297 - lava_callback - INFO - Received callback for node 69c70bb402375b4506dfc46e INFO: 172.20.0.1:60332 - "POST /node/69c70bb402375b4506dfc46e HTTP/1.0" 202 Accepted 2026-03-29 22:59:20,303 - lava_callback - INFO - Starting async job submit for node 69c70bb402375b4506dfc46e 2026-03-29 22:59:20,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:59:20,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70bb402375b4506dfc46e HTTP/1.1" 200 None 2026-03-29 22:59:20,329 - lava_callback - DEBUG - Processing node 69c70bb402375b4506dfc46e: baseline-arm-clabbe 2026-03-29 22:59:20,329 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70bb402375b4506dfc46e 2026-03-29 22:59:20,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:59:20,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70bb402375b4506dfc46e HTTP/1.1" 200 None 2026-03-29 22:59:20,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:59:20,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 22:59:20,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 22:59:20,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70bb402375b4506dfc46e HTTP/1.1" 200 None 2026-03-29 22:59:20,390 - lava_callback - ERROR - Error processing callback for node 69c70bb402375b4506dfc46e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49682 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:00:00,272 - lava_callback - INFO - Received callback for node 69c70bdc02375b4506dfc47e INFO: 172.20.0.1:57416 - "POST /node/69c70bdc02375b4506dfc47e HTTP/1.0" 202 Accepted 2026-03-29 23:00:00,278 - lava_callback - INFO - Starting async job submit for node 69c70bdc02375b4506dfc47e 2026-03-29 23:00:00,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:00:00,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70bdc02375b4506dfc47e HTTP/1.1" 200 None 2026-03-29 23:00:00,296 - lava_callback - DEBUG - Processing node 69c70bdc02375b4506dfc47e: baseline-arm-clabbe 2026-03-29 23:00:00,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70bdc02375b4506dfc47e 2026-03-29 23:00:00,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:00:00,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70bdc02375b4506dfc47e HTTP/1.1" 200 None 2026-03-29 23:00:00,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:00:00,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 23:00:00,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:00:00,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70bdc02375b4506dfc47e HTTP/1.1" 200 None 2026-03-29 23:00:00,358 - lava_callback - ERROR - Error processing callback for node 69c70bdc02375b4506dfc47e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46902 - "GET /node/69bdf4d2d54b5122738c6aa5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46914 - "GET /node/693a8132e17cfbe560b12c72%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET /node/698c4e75755d7811a4141109%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44096 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:06:00,604 - lava_callback - INFO - Received callback for node 69c70d4502375b4506dfc4ec INFO: 172.20.0.1:47718 - "POST /node/69c70d4502375b4506dfc4ec HTTP/1.0" 202 Accepted 2026-03-29 23:06:00,609 - lava_callback - INFO - Starting async job submit for node 69c70d4502375b4506dfc4ec 2026-03-29 23:06:00,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:00,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70d4502375b4506dfc4ec HTTP/1.1" 200 None 2026-03-29 23:06:00,625 - lava_callback - DEBUG - Processing node 69c70d4502375b4506dfc4ec: baseline-arm-clabbe 2026-03-29 23:06:00,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70d4502375b4506dfc4ec 2026-03-29 23:06:00,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:00,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70d4502375b4506dfc4ec HTTP/1.1" 200 None 2026-03-29 23:06:00,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:00,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:06:00,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:00,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70d4502375b4506dfc4ec HTTP/1.1" 200 None 2026-03-29 23:06:00,678 - lava_callback - ERROR - Error processing callback for node 69c70d4502375b4506dfc4ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58934 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:06:37,133 - lava_callback - INFO - Received callback for node 69c70d7202375b4506dfc4f9 INFO: 172.20.0.1:58948 - "POST /node/69c70d7202375b4506dfc4f9 HTTP/1.0" 202 Accepted 2026-03-29 23:06:37,139 - lava_callback - INFO - Starting async job submit for node 69c70d7202375b4506dfc4f9 2026-03-29 23:06:37,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:37,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70d7202375b4506dfc4f9 HTTP/1.1" 200 None 2026-03-29 23:06:37,156 - lava_callback - DEBUG - Processing node 69c70d7202375b4506dfc4f9: baseline-arm64-baylibre 2026-03-29 23:06:37,156 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70d7202375b4506dfc4f9 2026-03-29 23:06:37,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:37,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70d7202375b4506dfc4f9 HTTP/1.1" 200 None 2026-03-29 23:06:37,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:37,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:06:37,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:06:37,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70d7202375b4506dfc4f9 HTTP/1.1" 200 None 2026-03-29 23:06:37,208 - lava_callback - ERROR - Error processing callback for node 69c70d7202375b4506dfc4f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53668 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:08:00,655 - lava_callback - INFO - Received callback for node 69c70dbc02375b4506dfc507 INFO: 172.20.0.1:49194 - "POST /node/69c70dbc02375b4506dfc507 HTTP/1.0" 202 Accepted 2026-03-29 23:08:00,660 - lava_callback - INFO - Starting async job submit for node 69c70dbc02375b4506dfc507 2026-03-29 23:08:00,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:00,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70dbc02375b4506dfc507 HTTP/1.1" 200 None 2026-03-29 23:08:00,681 - lava_callback - DEBUG - Processing node 69c70dbc02375b4506dfc507: baseline-arm-clabbe 2026-03-29 23:08:00,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70dbc02375b4506dfc507 2026-03-29 23:08:00,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:00,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70dbc02375b4506dfc507 HTTP/1.1" 200 None 2026-03-29 23:08:00,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:00,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:08:00,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:00,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70dbc02375b4506dfc507 HTTP/1.1" 200 None 2026-03-29 23:08:00,738 - lava_callback - ERROR - Error processing callback for node 69c70dbc02375b4506dfc507: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:08:37,236 - lava_callback - INFO - Received callback for node 69c70de802375b4506dfc514 INFO: 172.20.0.1:55148 - "POST /node/69c70de802375b4506dfc514 HTTP/1.0" 202 Accepted 2026-03-29 23:08:37,241 - lava_callback - INFO - Starting async job submit for node 69c70de802375b4506dfc514 2026-03-29 23:08:37,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:37,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70de802375b4506dfc514 HTTP/1.1" 200 None 2026-03-29 23:08:37,257 - lava_callback - DEBUG - Processing node 69c70de802375b4506dfc514: baseline-arm64-baylibre 2026-03-29 23:08:37,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70de802375b4506dfc514 2026-03-29 23:08:37,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:37,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70de802375b4506dfc514 HTTP/1.1" 200 None 2026-03-29 23:08:37,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:37,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:08:37,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:08:37,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70de802375b4506dfc514 HTTP/1.1" 200 None 2026-03-29 23:08:37,309 - lava_callback - ERROR - Error processing callback for node 69c70de802375b4506dfc514: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52634 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:09:57,298 - lava_callback - INFO - Received callback for node 69c70e3702375b4506dfc520 INFO: 172.20.0.1:52646 - "POST /node/69c70e3702375b4506dfc520 HTTP/1.0" 202 Accepted 2026-03-29 23:09:57,307 - lava_callback - INFO - Starting async job submit for node 69c70e3702375b4506dfc520 2026-03-29 23:09:57,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:09:57,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70e3702375b4506dfc520 HTTP/1.1" 200 None 2026-03-29 23:09:57,322 - lava_callback - DEBUG - Processing node 69c70e3702375b4506dfc520: baseline-arm64-baylibre 2026-03-29 23:09:57,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70e3702375b4506dfc520 2026-03-29 23:09:57,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:09:57,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70e3702375b4506dfc520 HTTP/1.1" 200 None 2026-03-29 23:09:57,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:09:57,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:09:57,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:09:57,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70e3702375b4506dfc520 HTTP/1.1" 200 None 2026-03-29 23:09:57,388 - lava_callback - ERROR - Error processing callback for node 69c70e3702375b4506dfc520: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:11:17,396 - lava_callback - INFO - Received callback for node 69c70e8802375b4506dfc52a INFO: 172.20.0.1:58372 - "POST /node/69c70e8802375b4506dfc52a HTTP/1.0" 202 Accepted 2026-03-29 23:11:17,401 - lava_callback - INFO - Starting async job submit for node 69c70e8802375b4506dfc52a 2026-03-29 23:11:17,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:11:17,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70e8802375b4506dfc52a HTTP/1.1" 200 None 2026-03-29 23:11:17,417 - lava_callback - DEBUG - Processing node 69c70e8802375b4506dfc52a: baseline-arm64-baylibre 2026-03-29 23:11:17,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70e8802375b4506dfc52a 2026-03-29 23:11:17,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:11:17,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70e8802375b4506dfc52a HTTP/1.1" 200 None 2026-03-29 23:11:17,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:11:17,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:11:17,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:11:17,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70e8802375b4506dfc52a HTTP/1.1" 200 None 2026-03-29 23:11:17,474 - lava_callback - ERROR - Error processing callback for node 69c70e8802375b4506dfc52a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41986 - "GET /node/699197ba10b45da7d7d11e31%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET /node/69870f9e6e077e4126a7edcd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57224 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:12:37,459 - lava_callback - INFO - Received callback for node 69c70ed902375b4506dfc532 INFO: 172.20.0.1:57240 - "POST /node/69c70ed902375b4506dfc532 HTTP/1.0" 202 Accepted 2026-03-29 23:12:37,465 - lava_callback - INFO - Starting async job submit for node 69c70ed902375b4506dfc532 2026-03-29 23:12:37,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:12:37,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70ed902375b4506dfc532 HTTP/1.1" 200 None 2026-03-29 23:12:37,481 - lava_callback - DEBUG - Processing node 69c70ed902375b4506dfc532: baseline-arm64-baylibre 2026-03-29 23:12:37,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70ed902375b4506dfc532 2026-03-29 23:12:37,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:12:37,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70ed902375b4506dfc532 HTTP/1.1" 200 None 2026-03-29 23:12:37,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:12:37,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:12:37,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:12:37,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70ed902375b4506dfc532 HTTP/1.1" 200 None 2026-03-29 23:12:37,540 - lava_callback - ERROR - Error processing callback for node 69c70ed902375b4506dfc532: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET /node/698e24c3952a14bf500e3de4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36282 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:13:57,615 - lava_callback - INFO - Received callback for node 69c70f2a02375b4506dfc53e INFO: 172.20.0.1:36290 - "POST /node/69c70f2a02375b4506dfc53e HTTP/1.0" 202 Accepted 2026-03-29 23:13:57,620 - lava_callback - INFO - Starting async job submit for node 69c70f2a02375b4506dfc53e 2026-03-29 23:13:57,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:13:57,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70f2a02375b4506dfc53e HTTP/1.1" 200 None 2026-03-29 23:13:57,636 - lava_callback - DEBUG - Processing node 69c70f2a02375b4506dfc53e: baseline-arm64-baylibre 2026-03-29 23:13:57,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70f2a02375b4506dfc53e 2026-03-29 23:13:57,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:13:57,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70f2a02375b4506dfc53e HTTP/1.1" 200 None 2026-03-29 23:13:57,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:13:57,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:13:57,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:13:57,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70f2a02375b4506dfc53e HTTP/1.1" 200 None 2026-03-29 23:13:57,693 - lava_callback - ERROR - Error processing callback for node 69c70f2a02375b4506dfc53e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48162 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:15:21,036 - lava_callback - INFO - Received callback for node 69c70f7202375b4506dfc546 INFO: 172.20.0.1:50360 - "POST /node/69c70f7202375b4506dfc546 HTTP/1.0" 202 Accepted 2026-03-29 23:15:21,042 - lava_callback - INFO - Starting async job submit for node 69c70f7202375b4506dfc546 2026-03-29 23:15:21,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:21,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70f7202375b4506dfc546 HTTP/1.1" 200 None 2026-03-29 23:15:21,058 - lava_callback - DEBUG - Processing node 69c70f7202375b4506dfc546: baseline-arm-clabbe 2026-03-29 23:15:21,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70f7202375b4506dfc546 2026-03-29 23:15:21,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:21,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70f7202375b4506dfc546 HTTP/1.1" 200 None 2026-03-29 23:15:21,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:21,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 23:15:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:21,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70f7202375b4506dfc546 HTTP/1.1" 200 None 2026-03-29 23:15:21,117 - lava_callback - ERROR - Error processing callback for node 69c70f7202375b4506dfc546: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50148 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:15:41,025 - lava_callback - INFO - Received callback for node 69c70f9702375b4506dfc557 INFO: 172.20.0.1:55528 - "POST /node/69c70f9702375b4506dfc557 HTTP/1.0" 202 Accepted 2026-03-29 23:15:41,031 - lava_callback - INFO - Starting async job submit for node 69c70f9702375b4506dfc557 2026-03-29 23:15:41,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:41,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70f9702375b4506dfc557 HTTP/1.1" 200 None 2026-03-29 23:15:41,047 - lava_callback - DEBUG - Processing node 69c70f9702375b4506dfc557: baseline-arm-clabbe 2026-03-29 23:15:41,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70f9702375b4506dfc557 2026-03-29 23:15:41,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:41,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70f9702375b4506dfc557 HTTP/1.1" 200 None 2026-03-29 23:15:41,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:41,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-29 23:15:41,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:15:41,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70f9702375b4506dfc557 HTTP/1.1" 200 None 2026-03-29 23:15:41,121 - lava_callback - ERROR - Error processing callback for node 69c70f9702375b4506dfc557: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55542 - "GET /node/6982acd63507dcddca590654%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44392 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:16:41,125 - lava_callback - INFO - Received callback for node 69c70fca02375b4506dfc563 INFO: 172.20.0.1:49088 - "POST /node/69c70fca02375b4506dfc563 HTTP/1.0" 202 Accepted 2026-03-29 23:16:41,131 - lava_callback - INFO - Starting async job submit for node 69c70fca02375b4506dfc563 2026-03-29 23:16:41,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:16:41,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70fca02375b4506dfc563 HTTP/1.1" 200 None 2026-03-29 23:16:41,148 - lava_callback - DEBUG - Processing node 69c70fca02375b4506dfc563: baseline-arm64-clabbe 2026-03-29 23:16:41,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c70fca02375b4506dfc563 2026-03-29 23:16:41,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:16:41,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c70fca02375b4506dfc563 HTTP/1.1" 200 None 2026-03-29 23:16:41,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:16:41,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:16:41,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:16:41,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c70fca02375b4506dfc563 HTTP/1.1" 200 None 2026-03-29 23:16:41,203 - lava_callback - ERROR - Error processing callback for node 69c70fca02375b4506dfc563: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53678 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:18:01,146 - lava_callback - INFO - Received callback for node 69c7101902375b4506dfc56d INFO: 172.20.0.1:48850 - "POST /node/69c7101902375b4506dfc56d HTTP/1.0" 202 Accepted 2026-03-29 23:18:01,152 - lava_callback - INFO - Starting async job submit for node 69c7101902375b4506dfc56d 2026-03-29 23:18:01,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:18:01,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7101902375b4506dfc56d HTTP/1.1" 200 None 2026-03-29 23:18:01,168 - lava_callback - DEBUG - Processing node 69c7101902375b4506dfc56d: baseline-arm64-clabbe 2026-03-29 23:18:01,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7101902375b4506dfc56d 2026-03-29 23:18:01,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:18:01,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7101902375b4506dfc56d HTTP/1.1" 200 None 2026-03-29 23:18:01,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:18:01,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:18:01,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:18:01,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7101902375b4506dfc56d HTTP/1.1" 200 None 2026-03-29 23:18:01,240 - lava_callback - ERROR - Error processing callback for node 69c7101902375b4506dfc56d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40228 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:19:21,234 - lava_callback - INFO - Received callback for node 69c7106b02375b4506dfc57a INFO: 172.20.0.1:49358 - "POST /node/69c7106b02375b4506dfc57a HTTP/1.0" 202 Accepted 2026-03-29 23:19:21,239 - lava_callback - INFO - Starting async job submit for node 69c7106b02375b4506dfc57a 2026-03-29 23:19:21,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:19:21,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7106b02375b4506dfc57a HTTP/1.1" 200 None 2026-03-29 23:19:21,256 - lava_callback - DEBUG - Processing node 69c7106b02375b4506dfc57a: baseline-arm64-clabbe 2026-03-29 23:19:21,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7106b02375b4506dfc57a 2026-03-29 23:19:21,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:19:21,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7106b02375b4506dfc57a HTTP/1.1" 200 None 2026-03-29 23:19:21,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:19:21,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:19:21,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:19:21,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7106b02375b4506dfc57a HTTP/1.1" 200 None 2026-03-29 23:19:21,315 - lava_callback - ERROR - Error processing callback for node 69c7106b02375b4506dfc57a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:20:41,514 - lava_callback - INFO - Received callback for node 69c710bb02375b4506dfc57f INFO: 172.20.0.1:54870 - "POST /node/69c710bb02375b4506dfc57f HTTP/1.0" 202 Accepted 2026-03-29 23:20:41,520 - lava_callback - INFO - Starting async job submit for node 69c710bb02375b4506dfc57f 2026-03-29 23:20:41,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:20:41,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c710bb02375b4506dfc57f HTTP/1.1" 200 None 2026-03-29 23:20:41,536 - lava_callback - DEBUG - Processing node 69c710bb02375b4506dfc57f: baseline-arm64-clabbe 2026-03-29 23:20:41,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c710bb02375b4506dfc57f 2026-03-29 23:20:41,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:20:41,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c710bb02375b4506dfc57f HTTP/1.1" 200 None 2026-03-29 23:20:41,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:20:41,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:20:41,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:20:41,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c710bb02375b4506dfc57f HTTP/1.1" 200 None 2026-03-29 23:20:41,627 - lava_callback - ERROR - Error processing callback for node 69c710bb02375b4506dfc57f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35850 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:22:01,353 - lava_callback - INFO - Received callback for node 69c7110b02375b4506dfc58c INFO: 172.20.0.1:51990 - "POST /node/69c7110b02375b4506dfc58c HTTP/1.0" 202 Accepted 2026-03-29 23:22:01,358 - lava_callback - INFO - Starting async job submit for node 69c7110b02375b4506dfc58c 2026-03-29 23:22:01,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:22:01,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7110b02375b4506dfc58c HTTP/1.1" 200 None 2026-03-29 23:22:01,373 - lava_callback - DEBUG - Processing node 69c7110b02375b4506dfc58c: baseline-arm64-clabbe 2026-03-29 23:22:01,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7110b02375b4506dfc58c 2026-03-29 23:22:01,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:22:01,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7110b02375b4506dfc58c HTTP/1.1" 200 None 2026-03-29 23:22:01,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:22:01,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:22:01,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:22:01,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7110b02375b4506dfc58c HTTP/1.1" 200 None 2026-03-29 23:22:01,439 - lava_callback - ERROR - Error processing callback for node 69c7110b02375b4506dfc58c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59848 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:23:21,417 - lava_callback - INFO - Received callback for node 69c7115b02375b4506dfc59a INFO: 172.20.0.1:60188 - "POST /node/69c7115b02375b4506dfc59a HTTP/1.0" 202 Accepted 2026-03-29 23:23:21,423 - lava_callback - INFO - Starting async job submit for node 69c7115b02375b4506dfc59a 2026-03-29 23:23:21,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:23:21,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7115b02375b4506dfc59a HTTP/1.1" 200 None 2026-03-29 23:23:21,439 - lava_callback - DEBUG - Processing node 69c7115b02375b4506dfc59a: baseline-arm64-clabbe 2026-03-29 23:23:21,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7115b02375b4506dfc59a 2026-03-29 23:23:21,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:23:21,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7115b02375b4506dfc59a HTTP/1.1" 200 None 2026-03-29 23:23:21,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:23:21,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-29 23:23:21,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:23:21,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7115b02375b4506dfc59a HTTP/1.1" 200 None 2026-03-29 23:23:21,500 - lava_callback - ERROR - Error processing callback for node 69c7115b02375b4506dfc59a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52912 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:24:38,215 - lava_callback - INFO - Received callback for node 69c711a202375b4506dfc59f INFO: 172.20.0.1:52926 - "POST /node/69c711a202375b4506dfc59f HTTP/1.0" 202 Accepted 2026-03-29 23:24:38,224 - lava_callback - INFO - Starting async job submit for node 69c711a202375b4506dfc59f 2026-03-29 23:24:38,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:24:38,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c711a202375b4506dfc59f HTTP/1.1" 200 None 2026-03-29 23:24:38,259 - lava_callback - DEBUG - Processing node 69c711a202375b4506dfc59f: baseline-arm64-baylibre 2026-03-29 23:24:38,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c711a202375b4506dfc59f 2026-03-29 23:24:38,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:24:38,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c711a202375b4506dfc59f HTTP/1.1" 200 None 2026-03-29 23:24:38,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:24:38,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:24:38,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:24:38,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c711a202375b4506dfc59f HTTP/1.1" 200 None 2026-03-29 23:24:38,364 - lava_callback - ERROR - Error processing callback for node 69c711a202375b4506dfc59f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50558 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:25:38,088 - lava_callback - INFO - Received callback for node 69c711e902375b4506dfc5a8 INFO: 172.20.0.1:50566 - "POST /node/69c711e902375b4506dfc5a8 HTTP/1.0" 202 Accepted 2026-03-29 23:25:38,094 - lava_callback - INFO - Starting async job submit for node 69c711e902375b4506dfc5a8 2026-03-29 23:25:38,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:25:38,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c711e902375b4506dfc5a8 HTTP/1.1" 200 None 2026-03-29 23:25:38,110 - lava_callback - DEBUG - Processing node 69c711e902375b4506dfc5a8: baseline-arm-baylibre 2026-03-29 23:25:38,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c711e902375b4506dfc5a8 2026-03-29 23:25:38,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:25:38,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c711e902375b4506dfc5a8 HTTP/1.1" 200 None 2026-03-29 23:25:38,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:25:38,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:25:38,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:25:38,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c711e902375b4506dfc5a8 HTTP/1.1" 200 None 2026-03-29 23:25:38,165 - lava_callback - ERROR - Error processing callback for node 69c711e902375b4506dfc5a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:26:38,208 - lava_callback - INFO - Received callback for node 69c7121902375b4506dfc5b7 INFO: 172.20.0.1:42326 - "POST /node/69c7121902375b4506dfc5b7 HTTP/1.0" 202 Accepted 2026-03-29 23:26:38,214 - lava_callback - INFO - Starting async job submit for node 69c7121902375b4506dfc5b7 2026-03-29 23:26:38,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:26:38,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7121902375b4506dfc5b7 HTTP/1.1" 200 None 2026-03-29 23:26:38,232 - lava_callback - DEBUG - Processing node 69c7121902375b4506dfc5b7: baseline-arm64-baylibre 2026-03-29 23:26:38,232 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7121902375b4506dfc5b7 2026-03-29 23:26:38,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:26:38,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7121902375b4506dfc5b7 HTTP/1.1" 200 None 2026-03-29 23:26:38,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:26:38,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:26:38,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:26:38,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7121902375b4506dfc5b7 HTTP/1.1" 200 None 2026-03-29 23:26:38,285 - lava_callback - ERROR - Error processing callback for node 69c7121902375b4506dfc5b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48644 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:27:58,216 - lava_callback - INFO - Received callback for node 69c7126b02375b4506dfc5c6 INFO: 172.20.0.1:48658 - "POST /node/69c7126b02375b4506dfc5c6 HTTP/1.0" 202 Accepted 2026-03-29 23:27:58,223 - lava_callback - INFO - Starting async job submit for node 69c7126b02375b4506dfc5c6 2026-03-29 23:27:58,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:27:58,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7126b02375b4506dfc5c6 HTTP/1.1" 200 None 2026-03-29 23:27:58,249 - lava_callback - DEBUG - Processing node 69c7126b02375b4506dfc5c6: baseline-arm64-baylibre 2026-03-29 23:27:58,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7126b02375b4506dfc5c6 2026-03-29 23:27:58,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:27:58,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7126b02375b4506dfc5c6 HTTP/1.1" 200 None 2026-03-29 23:27:58,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:27:58,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:27:58,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:27:58,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7126b02375b4506dfc5c6 HTTP/1.1" 200 None 2026-03-29 23:27:58,303 - lava_callback - ERROR - Error processing callback for node 69c7126b02375b4506dfc5c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34746 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:29:01,524 - lava_callback - INFO - Received callback for node 69c712b302375b4506dfc5cd INFO: 172.20.0.1:55570 - "POST /node/69c712b302375b4506dfc5cd HTTP/1.0" 202 Accepted 2026-03-29 23:29:01,529 - lava_callback - INFO - Starting async job submit for node 69c712b302375b4506dfc5cd 2026-03-29 23:29:01,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:01,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c712b302375b4506dfc5cd HTTP/1.1" 200 None 2026-03-29 23:29:01,546 - lava_callback - DEBUG - Processing node 69c712b302375b4506dfc5cd: baseline-arm-clabbe 2026-03-29 23:29:01,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c712b302375b4506dfc5cd 2026-03-29 23:29:01,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:01,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c712b302375b4506dfc5cd HTTP/1.1" 200 None 2026-03-29 23:29:01,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:01,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:29:01,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:01,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c712b302375b4506dfc5cd HTTP/1.1" 200 None 2026-03-29 23:29:01,607 - lava_callback - ERROR - Error processing callback for node 69c712b302375b4506dfc5cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37940 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:29:41,579 - lava_callback - INFO - Received callback for node 69c712db02375b4506dfc5da INFO: 172.20.0.1:36312 - "POST /node/69c712db02375b4506dfc5da HTTP/1.0" 202 Accepted 2026-03-29 23:29:41,584 - lava_callback - INFO - Starting async job submit for node 69c712db02375b4506dfc5da 2026-03-29 23:29:41,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:41,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c712db02375b4506dfc5da HTTP/1.1" 200 None 2026-03-29 23:29:41,601 - lava_callback - DEBUG - Processing node 69c712db02375b4506dfc5da: baseline-arm-clabbe 2026-03-29 23:29:41,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c712db02375b4506dfc5da 2026-03-29 23:29:41,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:41,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c712db02375b4506dfc5da HTTP/1.1" 200 None 2026-03-29 23:29:41,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:41,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:29:41,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:29:41,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c712db02375b4506dfc5da HTTP/1.1" 200 None 2026-03-29 23:29:41,656 - lava_callback - ERROR - Error processing callback for node 69c712db02375b4506dfc5da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46090 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:30:21,577 - lava_callback - INFO - Received callback for node 69c7130202375b4506dfc5ea INFO: 172.20.0.1:39602 - "POST /node/69c7130202375b4506dfc5ea HTTP/1.0" 202 Accepted 2026-03-29 23:30:21,583 - lava_callback - INFO - Starting async job submit for node 69c7130202375b4506dfc5ea 2026-03-29 23:30:21,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:21,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7130202375b4506dfc5ea HTTP/1.1" 200 None 2026-03-29 23:30:21,599 - lava_callback - DEBUG - Processing node 69c7130202375b4506dfc5ea: baseline-arm-clabbe 2026-03-29 23:30:21,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7130202375b4506dfc5ea 2026-03-29 23:30:21,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:21,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7130202375b4506dfc5ea HTTP/1.1" 200 None 2026-03-29 23:30:21,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:21,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:30:21,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:21,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7130202375b4506dfc5ea HTTP/1.1" 200 None 2026-03-29 23:30:21,660 - lava_callback - ERROR - Error processing callback for node 69c7130202375b4506dfc5ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:30:58,410 - lava_callback - INFO - Received callback for node 69c7133002375b4506dfc5f8 INFO: 172.20.0.1:57704 - "POST /node/69c7133002375b4506dfc5f8 HTTP/1.0" 202 Accepted 2026-03-29 23:30:58,416 - lava_callback - INFO - Starting async job submit for node 69c7133002375b4506dfc5f8 2026-03-29 23:30:58,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:58,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7133002375b4506dfc5f8 HTTP/1.1" 200 None 2026-03-29 23:30:58,432 - lava_callback - DEBUG - Processing node 69c7133002375b4506dfc5f8: baseline-arm64-baylibre 2026-03-29 23:30:58,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7133002375b4506dfc5f8 2026-03-29 23:30:58,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:58,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7133002375b4506dfc5f8 HTTP/1.1" 200 None 2026-03-29 23:30:58,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:58,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:30:58,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:30:58,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7133002375b4506dfc5f8 HTTP/1.1" 200 None 2026-03-29 23:30:58,485 - lava_callback - ERROR - Error processing callback for node 69c7133002375b4506dfc5f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:32:21,651 - lava_callback - INFO - Received callback for node 69c7137802375b4506dfc607 INFO: 172.20.0.1:57334 - "POST /node/69c7137802375b4506dfc607 HTTP/1.0" 202 Accepted 2026-03-29 23:32:21,656 - lava_callback - INFO - Starting async job submit for node 69c7137802375b4506dfc607 2026-03-29 23:32:21,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:32:21,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7137802375b4506dfc607 HTTP/1.1" 200 None 2026-03-29 23:32:21,697 - lava_callback - DEBUG - Processing node 69c7137802375b4506dfc607: baseline-arm-clabbe 2026-03-29 23:32:21,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7137802375b4506dfc607 2026-03-29 23:32:21,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:32:21,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7137802375b4506dfc607 HTTP/1.1" 200 None 2026-03-29 23:32:21,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:32:21,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:32:21,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:32:21,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7137802375b4506dfc607 HTTP/1.1" 200 None 2026-03-29 23:32:21,749 - lava_callback - ERROR - Error processing callback for node 69c7137802375b4506dfc607: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43838 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:33:21,679 - lava_callback - INFO - Received callback for node 69c713ae02375b4506dfc616 INFO: 172.20.0.1:35926 - "POST /node/69c713ae02375b4506dfc616 HTTP/1.0" 202 Accepted 2026-03-29 23:33:21,684 - lava_callback - INFO - Starting async job submit for node 69c713ae02375b4506dfc616 2026-03-29 23:33:21,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:33:21,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c713ae02375b4506dfc616 HTTP/1.1" 200 None 2026-03-29 23:33:21,700 - lava_callback - DEBUG - Processing node 69c713ae02375b4506dfc616: baseline-arm64-clabbe 2026-03-29 23:33:21,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c713ae02375b4506dfc616 2026-03-29 23:33:21,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:33:21,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c713ae02375b4506dfc616 HTTP/1.1" 200 None 2026-03-29 23:33:21,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:33:21,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:33:21,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:33:21,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c713ae02375b4506dfc616 HTTP/1.1" 200 None 2026-03-29 23:33:21,771 - lava_callback - ERROR - Error processing callback for node 69c713ae02375b4506dfc616: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55392 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:35:41,847 - lava_callback - INFO - Received callback for node 69c7143c02375b4506dfc626 INFO: 172.20.0.1:35766 - "POST /node/69c7143c02375b4506dfc626 HTTP/1.0" 202 Accepted 2026-03-29 23:35:41,852 - lava_callback - INFO - Starting async job submit for node 69c7143c02375b4506dfc626 2026-03-29 23:35:41,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:35:41,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7143c02375b4506dfc626 HTTP/1.1" 200 None 2026-03-29 23:35:41,877 - lava_callback - DEBUG - Processing node 69c7143c02375b4506dfc626: baseline-arm64-clabbe 2026-03-29 23:35:41,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7143c02375b4506dfc626 2026-03-29 23:35:41,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:35:41,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7143c02375b4506dfc626 HTTP/1.1" 200 None 2026-03-29 23:35:41,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:35:41,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:35:41,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:35:41,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7143c02375b4506dfc626 HTTP/1.1" 200 None 2026-03-29 23:35:42,005 - lava_callback - ERROR - Error processing callback for node 69c7143c02375b4506dfc626: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57236 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:37:01,828 - lava_callback - INFO - Received callback for node 69c7148c02375b4506dfc633 INFO: 172.20.0.1:57676 - "POST /node/69c7148c02375b4506dfc633 HTTP/1.0" 202 Accepted 2026-03-29 23:37:01,834 - lava_callback - INFO - Starting async job submit for node 69c7148c02375b4506dfc633 2026-03-29 23:37:01,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:37:01,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7148c02375b4506dfc633 HTTP/1.1" 200 None 2026-03-29 23:37:01,852 - lava_callback - DEBUG - Processing node 69c7148c02375b4506dfc633: baseline-arm64-clabbe 2026-03-29 23:37:01,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7148c02375b4506dfc633 2026-03-29 23:37:01,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:37:01,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7148c02375b4506dfc633 HTTP/1.1" 200 None 2026-03-29 23:37:01,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:37:01,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:37:01,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:37:01,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7148c02375b4506dfc633 HTTP/1.1" 200 None 2026-03-29 23:37:01,905 - lava_callback - ERROR - Error processing callback for node 69c7148c02375b4506dfc633: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42600 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:39:18,859 - lava_callback - INFO - Received callback for node 69c7151302375b4506dfc652 INFO: 172.20.0.1:42604 - "POST /node/69c7151302375b4506dfc652 HTTP/1.0" 202 Accepted 2026-03-29 23:39:18,865 - lava_callback - INFO - Starting async job submit for node 69c7151302375b4506dfc652 2026-03-29 23:39:18,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:39:18,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7151302375b4506dfc652 HTTP/1.1" 200 None 2026-03-29 23:39:18,880 - lava_callback - DEBUG - Processing node 69c7151302375b4506dfc652: baseline-arm-baylibre 2026-03-29 23:39:18,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7151302375b4506dfc652 2026-03-29 23:39:18,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:39:18,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7151302375b4506dfc652 HTTP/1.1" 200 None 2026-03-29 23:39:18,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:39:18,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:39:18,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:39:18,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7151302375b4506dfc652 HTTP/1.1" 200 None 2026-03-29 23:39:18,932 - lava_callback - ERROR - Error processing callback for node 69c7151302375b4506dfc652: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49118 - "GET /node/69bd096a05e4bbd6ba635701%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49516 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:40:59,068 - lava_callback - INFO - Received callback for node 69c7158402375b4506dfc674 INFO: 172.20.0.1:49522 - "POST /node/69c7158402375b4506dfc674 HTTP/1.0" 202 Accepted 2026-03-29 23:40:59,073 - lava_callback - INFO - Starting async job submit for node 69c7158402375b4506dfc674 2026-03-29 23:40:59,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:40:59,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7158402375b4506dfc674 HTTP/1.1" 200 None 2026-03-29 23:40:59,090 - lava_callback - DEBUG - Processing node 69c7158402375b4506dfc674: baseline-arm-baylibre 2026-03-29 23:40:59,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7158402375b4506dfc674 2026-03-29 23:40:59,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:40:59,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7158402375b4506dfc674 HTTP/1.1" 200 None 2026-03-29 23:40:59,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:40:59,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:40:59,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:40:59,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7158402375b4506dfc674 HTTP/1.1" 200 None 2026-03-29 23:40:59,170 - lava_callback - ERROR - Error processing callback for node 69c7158402375b4506dfc674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:41:38,916 - lava_callback - INFO - Received callback for node 69c715ab02375b4506dfc681 INFO: 172.20.0.1:34198 - "POST /node/69c715ab02375b4506dfc681 HTTP/1.0" 202 Accepted 2026-03-29 23:41:38,921 - lava_callback - INFO - Starting async job submit for node 69c715ab02375b4506dfc681 2026-03-29 23:41:38,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:41:38,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c715ab02375b4506dfc681 HTTP/1.1" 200 None 2026-03-29 23:41:38,938 - lava_callback - DEBUG - Processing node 69c715ab02375b4506dfc681: baseline-arm-baylibre 2026-03-29 23:41:38,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c715ab02375b4506dfc681 2026-03-29 23:41:38,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:41:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c715ab02375b4506dfc681 HTTP/1.1" 200 None 2026-03-29 23:41:38,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:41:38,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:41:38,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:41:39,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c715ab02375b4506dfc681 HTTP/1.1" 200 None 2026-03-29 23:41:39,003 - lava_callback - ERROR - Error processing callback for node 69c715ab02375b4506dfc681: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49524 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:44:22,171 - lava_callback - INFO - Received callback for node 69c7164302375b4506dfc6bf INFO: 172.20.0.1:38702 - "POST /node/69c7164302375b4506dfc6bf HTTP/1.0" 202 Accepted 2026-03-29 23:44:22,177 - lava_callback - INFO - Starting async job submit for node 69c7164302375b4506dfc6bf 2026-03-29 23:44:22,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:44:22,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7164302375b4506dfc6bf HTTP/1.1" 200 None 2026-03-29 23:44:22,196 - lava_callback - DEBUG - Processing node 69c7164302375b4506dfc6bf: baseline-arm-clabbe 2026-03-29 23:44:22,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7164302375b4506dfc6bf 2026-03-29 23:44:22,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:44:22,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7164302375b4506dfc6bf HTTP/1.1" 200 None 2026-03-29 23:44:22,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:44:22,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:44:22,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:44:22,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7164302375b4506dfc6bf HTTP/1.1" 200 None 2026-03-29 23:44:22,261 - lava_callback - ERROR - Error processing callback for node 69c7164302375b4506dfc6bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:45:02,072 - lava_callback - INFO - Received callback for node 69c7166a02375b4506dfc6c8 INFO: 172.20.0.1:50142 - "POST /node/69c7166a02375b4506dfc6c8 HTTP/1.0" 202 Accepted 2026-03-29 23:45:02,078 - lava_callback - INFO - Starting async job submit for node 69c7166a02375b4506dfc6c8 2026-03-29 23:45:02,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:02,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7166a02375b4506dfc6c8 HTTP/1.1" 200 None 2026-03-29 23:45:02,094 - lava_callback - DEBUG - Processing node 69c7166a02375b4506dfc6c8: baseline-arm-clabbe 2026-03-29 23:45:02,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7166a02375b4506dfc6c8 2026-03-29 23:45:02,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:02,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7166a02375b4506dfc6c8 HTTP/1.1" 200 None 2026-03-29 23:45:02,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:02,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:45:02,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:02,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7166a02375b4506dfc6c8 HTTP/1.1" 200 None 2026-03-29 23:45:02,157 - lava_callback - ERROR - Error processing callback for node 69c7166a02375b4506dfc6c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40480 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:45:42,158 - lava_callback - INFO - Received callback for node 69c7169002375b4506dfc6d7 INFO: 172.20.0.1:50096 - "POST /node/69c7169002375b4506dfc6d7 HTTP/1.0" 202 Accepted 2026-03-29 23:45:42,163 - lava_callback - INFO - Starting async job submit for node 69c7169002375b4506dfc6d7 2026-03-29 23:45:42,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:42,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7169002375b4506dfc6d7 HTTP/1.1" 200 None 2026-03-29 23:45:42,180 - lava_callback - DEBUG - Processing node 69c7169002375b4506dfc6d7: baseline-arm-clabbe 2026-03-29 23:45:42,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7169002375b4506dfc6d7 2026-03-29 23:45:42,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:42,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7169002375b4506dfc6d7 HTTP/1.1" 200 None 2026-03-29 23:45:42,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:42,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:45:42,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:45:42,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7169002375b4506dfc6d7 HTTP/1.1" 200 None 2026-03-29 23:45:42,241 - lava_callback - ERROR - Error processing callback for node 69c7169002375b4506dfc6d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:46:02,136 - lava_callback - INFO - Received callback for node 69c716b602375b4506dfc6e5 INFO: 172.20.0.1:34012 - "POST /node/69c716b602375b4506dfc6e5 HTTP/1.0" 202 Accepted 2026-03-29 23:46:02,142 - lava_callback - INFO - Starting async job submit for node 69c716b602375b4506dfc6e5 2026-03-29 23:46:02,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:02,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c716b602375b4506dfc6e5 HTTP/1.1" 200 None 2026-03-29 23:46:02,162 - lava_callback - DEBUG - Processing node 69c716b602375b4506dfc6e5: baseline-arm-clabbe 2026-03-29 23:46:02,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c716b602375b4506dfc6e5 2026-03-29 23:46:02,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:02,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c716b602375b4506dfc6e5 HTTP/1.1" 200 None 2026-03-29 23:46:02,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:02,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:46:02,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:02,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c716b602375b4506dfc6e5 HTTP/1.1" 200 None 2026-03-29 23:46:02,228 - lava_callback - ERROR - Error processing callback for node 69c716b602375b4506dfc6e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:46:42,176 - lava_callback - INFO - Received callback for node 69c716dd02375b4506dfc6f8 INFO: 172.20.0.1:58420 - "POST /node/69c716dd02375b4506dfc6f8 HTTP/1.0" 202 Accepted 2026-03-29 23:46:42,182 - lava_callback - INFO - Starting async job submit for node 69c716dd02375b4506dfc6f8 2026-03-29 23:46:42,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:42,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c716dd02375b4506dfc6f8 HTTP/1.1" 200 None 2026-03-29 23:46:42,198 - lava_callback - DEBUG - Processing node 69c716dd02375b4506dfc6f8: baseline-arm-clabbe 2026-03-29 23:46:42,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c716dd02375b4506dfc6f8 2026-03-29 23:46:42,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:42,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c716dd02375b4506dfc6f8 HTTP/1.1" 200 None 2026-03-29 23:46:42,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:42,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-29 23:46:42,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:46:42,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c716dd02375b4506dfc6f8 HTTP/1.1" 200 None 2026-03-29 23:46:42,260 - lava_callback - ERROR - Error processing callback for node 69c716dd02375b4506dfc6f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51590 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:47:42,208 - lava_callback - INFO - Received callback for node 69c7171402375b4506dfc702 INFO: 172.20.0.1:59976 - "POST /node/69c7171402375b4506dfc702 HTTP/1.0" 202 Accepted 2026-03-29 23:47:42,213 - lava_callback - INFO - Starting async job submit for node 69c7171402375b4506dfc702 2026-03-29 23:47:42,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:47:42,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7171402375b4506dfc702 HTTP/1.1" 200 None 2026-03-29 23:47:42,230 - lava_callback - DEBUG - Processing node 69c7171402375b4506dfc702: baseline-arm64-clabbe 2026-03-29 23:47:42,230 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7171402375b4506dfc702 2026-03-29 23:47:42,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:47:42,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7171402375b4506dfc702 HTTP/1.1" 200 None 2026-03-29 23:47:42,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:47:42,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:47:42,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:47:42,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7171402375b4506dfc702 HTTP/1.1" 200 None 2026-03-29 23:47:42,282 - lava_callback - ERROR - Error processing callback for node 69c7171402375b4506dfc702: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:49:02,289 - lava_callback - INFO - Received callback for node 69c7176302375b4506dfc70e INFO: 172.20.0.1:44910 - "POST /node/69c7176302375b4506dfc70e HTTP/1.0" 202 Accepted 2026-03-29 23:49:02,294 - lava_callback - INFO - Starting async job submit for node 69c7176302375b4506dfc70e 2026-03-29 23:49:02,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:49:02,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7176302375b4506dfc70e HTTP/1.1" 200 None 2026-03-29 23:49:02,310 - lava_callback - DEBUG - Processing node 69c7176302375b4506dfc70e: baseline-arm64-clabbe 2026-03-29 23:49:02,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7176302375b4506dfc70e 2026-03-29 23:49:02,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:49:02,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7176302375b4506dfc70e HTTP/1.1" 200 None 2026-03-29 23:49:02,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:49:02,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:49:02,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:49:02,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7176302375b4506dfc70e HTTP/1.1" 200 None 2026-03-29 23:49:02,394 - lava_callback - ERROR - Error processing callback for node 69c7176302375b4506dfc70e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32812 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:50:02,370 - lava_callback - INFO - Received callback for node 69c717a502375b4506dfc716 INFO: 172.20.0.1:40718 - "POST /node/69c717a502375b4506dfc716 HTTP/1.0" 202 Accepted 2026-03-29 23:50:02,376 - lava_callback - INFO - Starting async job submit for node 69c717a502375b4506dfc716 2026-03-29 23:50:02,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:50:02,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c717a502375b4506dfc716 HTTP/1.1" 200 None 2026-03-29 23:50:02,394 - lava_callback - DEBUG - Processing node 69c717a502375b4506dfc716: baseline-arm-clabbe 2026-03-29 23:50:02,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c717a502375b4506dfc716 2026-03-29 23:50:02,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:50:02,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c717a502375b4506dfc716 HTTP/1.1" 200 None 2026-03-29 23:50:02,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:50:02,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:50:02,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:50:02,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c717a502375b4506dfc716 HTTP/1.1" 200 None 2026-03-29 23:50:02,471 - lava_callback - ERROR - Error processing callback for node 69c717a502375b4506dfc716: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42662 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:51:02,351 - lava_callback - INFO - Received callback for node 69c717db02375b4506dfc727 INFO: 172.20.0.1:57530 - "POST /node/69c717db02375b4506dfc727 HTTP/1.0" 202 Accepted 2026-03-29 23:51:02,357 - lava_callback - INFO - Starting async job submit for node 69c717db02375b4506dfc727 2026-03-29 23:51:02,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:51:02,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c717db02375b4506dfc727 HTTP/1.1" 200 None 2026-03-29 23:51:02,373 - lava_callback - DEBUG - Processing node 69c717db02375b4506dfc727: baseline-arm64-clabbe 2026-03-29 23:51:02,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c717db02375b4506dfc727 2026-03-29 23:51:02,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:51:02,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c717db02375b4506dfc727 HTTP/1.1" 200 None 2026-03-29 23:51:02,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:51:02,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:51:02,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:51:02,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c717db02375b4506dfc727 HTTP/1.1" 200 None 2026-03-29 23:51:02,427 - lava_callback - ERROR - Error processing callback for node 69c717db02375b4506dfc727: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34370 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:52:02,371 - lava_callback - INFO - Received callback for node 69c7181c02375b4506dfc72d INFO: 172.20.0.1:52736 - "POST /node/69c7181c02375b4506dfc72d HTTP/1.0" 202 Accepted 2026-03-29 23:52:02,377 - lava_callback - INFO - Starting async job submit for node 69c7181c02375b4506dfc72d 2026-03-29 23:52:02,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:02,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7181c02375b4506dfc72d HTTP/1.1" 200 None 2026-03-29 23:52:02,401 - lava_callback - DEBUG - Processing node 69c7181c02375b4506dfc72d: baseline-arm-clabbe 2026-03-29 23:52:02,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7181c02375b4506dfc72d 2026-03-29 23:52:02,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:02,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7181c02375b4506dfc72d HTTP/1.1" 200 None 2026-03-29 23:52:02,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:02,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:52:02,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:02,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7181c02375b4506dfc72d HTTP/1.1" 200 None 2026-03-29 23:52:02,471 - lava_callback - ERROR - Error processing callback for node 69c7181c02375b4506dfc72d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:52:42,345 - lava_callback - INFO - Received callback for node 69c7184202375b4506dfc743 INFO: 172.20.0.1:57674 - "POST /node/69c7184202375b4506dfc743 HTTP/1.0" 202 Accepted 2026-03-29 23:52:42,351 - lava_callback - INFO - Starting async job submit for node 69c7184202375b4506dfc743 2026-03-29 23:52:42,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:42,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7184202375b4506dfc743 HTTP/1.1" 200 None 2026-03-29 23:52:42,386 - lava_callback - DEBUG - Processing node 69c7184202375b4506dfc743: baseline-arm-clabbe 2026-03-29 23:52:42,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7184202375b4506dfc743 2026-03-29 23:52:42,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:42,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7184202375b4506dfc743 HTTP/1.1" 200 None 2026-03-29 23:52:42,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:42,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:52:42,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:52:42,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7184202375b4506dfc743 HTTP/1.1" 200 None 2026-03-29 23:52:42,446 - lava_callback - ERROR - Error processing callback for node 69c7184202375b4506dfc743: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48562 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:53:19,569 - lava_callback - INFO - Received callback for node 69c7186c02375b4506dfc74d INFO: 172.20.0.1:48564 - "POST /node/69c7186c02375b4506dfc74d HTTP/1.0" 202 Accepted 2026-03-29 23:53:19,574 - lava_callback - INFO - Starting async job submit for node 69c7186c02375b4506dfc74d 2026-03-29 23:53:19,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:53:19,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7186c02375b4506dfc74d HTTP/1.1" 200 None 2026-03-29 23:53:19,590 - lava_callback - DEBUG - Processing node 69c7186c02375b4506dfc74d: baseline-arm64-baylibre 2026-03-29 23:53:19,590 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7186c02375b4506dfc74d 2026-03-29 23:53:19,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:53:19,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7186c02375b4506dfc74d HTTP/1.1" 200 None 2026-03-29 23:53:19,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:53:19,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:53:19,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:53:19,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7186c02375b4506dfc74d HTTP/1.1" 200 None 2026-03-29 23:53:19,655 - lava_callback - ERROR - Error processing callback for node 69c7186c02375b4506dfc74d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33030 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:54:39,657 - lava_callback - INFO - Received callback for node 69c718bb02375b4506dfc758 INFO: 172.20.0.1:33038 - "POST /node/69c718bb02375b4506dfc758 HTTP/1.0" 202 Accepted 2026-03-29 23:54:39,663 - lava_callback - INFO - Starting async job submit for node 69c718bb02375b4506dfc758 2026-03-29 23:54:39,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:54:39,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c718bb02375b4506dfc758 HTTP/1.1" 200 None 2026-03-29 23:54:39,679 - lava_callback - DEBUG - Processing node 69c718bb02375b4506dfc758: baseline-arm64-baylibre 2026-03-29 23:54:39,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c718bb02375b4506dfc758 2026-03-29 23:54:39,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:54:39,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c718bb02375b4506dfc758 HTTP/1.1" 200 None 2026-03-29 23:54:39,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:54:39,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:54:39,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:54:39,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c718bb02375b4506dfc758 HTTP/1.1" 200 None 2026-03-29 23:54:39,743 - lava_callback - ERROR - Error processing callback for node 69c718bb02375b4506dfc758: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43792 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:55:59,828 - lava_callback - INFO - Received callback for node 69c7190c02375b4506dfc760 INFO: 172.20.0.1:40410 - "POST /node/69c7190c02375b4506dfc760 HTTP/1.0" 202 Accepted 2026-03-29 23:55:59,836 - lava_callback - INFO - Starting async job submit for node 69c7190c02375b4506dfc760 2026-03-29 23:55:59,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:55:59,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7190c02375b4506dfc760 HTTP/1.1" 200 None 2026-03-29 23:55:59,880 - lava_callback - DEBUG - Processing node 69c7190c02375b4506dfc760: baseline-arm64-baylibre 2026-03-29 23:55:59,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7190c02375b4506dfc760 2026-03-29 23:55:59,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:55:59,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7190c02375b4506dfc760 HTTP/1.1" 200 None 2026-03-29 23:55:59,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:55:59,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:55:59,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:55:59,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7190c02375b4506dfc760 HTTP/1.1" 200 None 2026-03-29 23:55:59,960 - lava_callback - ERROR - Error processing callback for node 69c7190c02375b4506dfc760: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45216 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:57:19,887 - lava_callback - INFO - Received callback for node 69c7195c02375b4506dfc76d INFO: 172.20.0.1:33588 - "POST /node/69c7195c02375b4506dfc76d HTTP/1.0" 202 Accepted 2026-03-29 23:57:19,892 - lava_callback - INFO - Starting async job submit for node 69c7195c02375b4506dfc76d 2026-03-29 23:57:19,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:57:19,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7195c02375b4506dfc76d HTTP/1.1" 200 None 2026-03-29 23:57:19,908 - lava_callback - DEBUG - Processing node 69c7195c02375b4506dfc76d: baseline-arm64-baylibre 2026-03-29 23:57:19,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c7195c02375b4506dfc76d 2026-03-29 23:57:19,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:57:19,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7195c02375b4506dfc76d HTTP/1.1" 200 None 2026-03-29 23:57:19,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:57:19,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:57:19,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:57:19,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7195c02375b4506dfc76d HTTP/1.1" 200 None 2026-03-29 23:57:19,987 - lava_callback - ERROR - Error processing callback for node 69c7195c02375b4506dfc76d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59010 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:58:42,703 - lava_callback - INFO - Received callback for node 69c719a402375b4506dfc778 INFO: 172.20.0.1:58742 - "POST /node/69c719a402375b4506dfc778 HTTP/1.0" 202 Accepted 2026-03-29 23:58:42,708 - lava_callback - INFO - Starting async job submit for node 69c719a402375b4506dfc778 2026-03-29 23:58:42,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:58:42,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c719a402375b4506dfc778 HTTP/1.1" 200 None 2026-03-29 23:58:42,724 - lava_callback - DEBUG - Processing node 69c719a402375b4506dfc778: baseline-arm-clabbe 2026-03-29 23:58:42,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c719a402375b4506dfc778 2026-03-29 23:58:42,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:58:42,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c719a402375b4506dfc778 HTTP/1.1" 200 None 2026-03-29 23:58:42,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:58:42,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-29 23:58:42,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:58:42,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c719a402375b4506dfc778 HTTP/1.1" 200 None 2026-03-29 23:58:42,782 - lava_callback - ERROR - Error processing callback for node 69c719a402375b4506dfc778: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46090 - "GET / HTTP/1.0" 200 OK 2026-03-29 23:59:42,705 - lava_callback - INFO - Received callback for node 69c719d802375b4506dfc78b INFO: 172.20.0.1:43676 - "POST /node/69c719d802375b4506dfc78b HTTP/1.0" 202 Accepted 2026-03-29 23:59:42,711 - lava_callback - INFO - Starting async job submit for node 69c719d802375b4506dfc78b 2026-03-29 23:59:42,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:59:42,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c719d802375b4506dfc78b HTTP/1.1" 200 None 2026-03-29 23:59:42,732 - lava_callback - DEBUG - Processing node 69c719d802375b4506dfc78b: baseline-arm64-clabbe 2026-03-29 23:59:42,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c719d802375b4506dfc78b 2026-03-29 23:59:42,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:59:42,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c719d802375b4506dfc78b HTTP/1.1" 200 None 2026-03-29 23:59:42,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:59:42,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-29 23:59:42,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 23:59:42,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c719d802375b4506dfc78b HTTP/1.1" 200 None 2026-03-29 23:59:42,791 - lava_callback - ERROR - Error processing callback for node 69c719d802375b4506dfc78b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40988 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:01:02,836 - lava_callback - INFO - Received callback for node 69c71a2e02375b4506dfc797 INFO: 172.20.0.1:41358 - "POST /node/69c71a2e02375b4506dfc797 HTTP/1.0" 202 Accepted 2026-03-30 00:01:02,843 - lava_callback - INFO - Starting async job submit for node 69c71a2e02375b4506dfc797 2026-03-30 00:01:02,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:01:02,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71a2e02375b4506dfc797 HTTP/1.1" 200 None 2026-03-30 00:01:02,865 - lava_callback - DEBUG - Processing node 69c71a2e02375b4506dfc797: baseline-arm64-clabbe 2026-03-30 00:01:02,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71a2e02375b4506dfc797 2026-03-30 00:01:02,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:01:02,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71a2e02375b4506dfc797 HTTP/1.1" 200 None 2026-03-30 00:01:02,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:01:02,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-30 00:01:02,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:01:02,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71a2e02375b4506dfc797 HTTP/1.1" 200 None 2026-03-30 00:01:02,938 - lava_callback - ERROR - Error processing callback for node 69c71a2e02375b4506dfc797: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:02:22,864 - lava_callback - INFO - Received callback for node 69c71a8202375b4506dfc7a4 INFO: 172.20.0.1:36550 - "POST /node/69c71a8202375b4506dfc7a4 HTTP/1.0" 202 Accepted 2026-03-30 00:02:22,871 - lava_callback - INFO - Starting async job submit for node 69c71a8202375b4506dfc7a4 2026-03-30 00:02:22,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:02:22,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71a8202375b4506dfc7a4 HTTP/1.1" 200 None 2026-03-30 00:02:22,887 - lava_callback - DEBUG - Processing node 69c71a8202375b4506dfc7a4: baseline-arm64-clabbe 2026-03-30 00:02:22,888 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71a8202375b4506dfc7a4 2026-03-30 00:02:22,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:02:22,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71a8202375b4506dfc7a4 HTTP/1.1" 200 None 2026-03-30 00:02:22,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:02:22,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-30 00:02:22,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:02:22,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71a8202375b4506dfc7a4 HTTP/1.1" 200 None 2026-03-30 00:02:22,946 - lava_callback - ERROR - Error processing callback for node 69c71a8202375b4506dfc7a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:03:42,922 - lava_callback - INFO - Received callback for node 69c71ad602375b4506dfc7ad INFO: 172.20.0.1:45074 - "POST /node/69c71ad602375b4506dfc7ad HTTP/1.0" 202 Accepted 2026-03-30 00:03:42,927 - lava_callback - INFO - Starting async job submit for node 69c71ad602375b4506dfc7ad 2026-03-30 00:03:42,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:03:42,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71ad602375b4506dfc7ad HTTP/1.1" 200 None 2026-03-30 00:03:42,944 - lava_callback - DEBUG - Processing node 69c71ad602375b4506dfc7ad: baseline-arm64-clabbe 2026-03-30 00:03:42,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71ad602375b4506dfc7ad 2026-03-30 00:03:42,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:03:42,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71ad602375b4506dfc7ad HTTP/1.1" 200 None 2026-03-30 00:03:42,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:03:42,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-30 00:03:42,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:03:43,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71ad602375b4506dfc7ad HTTP/1.1" 200 None 2026-03-30 00:03:43,006 - lava_callback - ERROR - Error processing callback for node 69c71ad602375b4506dfc7ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52364 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:05:02,904 - lava_callback - INFO - Received callback for node 69c71b2302375b4506dfc7b5 INFO: 172.20.0.1:58718 - "POST /node/69c71b2302375b4506dfc7b5 HTTP/1.0" 202 Accepted 2026-03-30 00:05:02,909 - lava_callback - INFO - Starting async job submit for node 69c71b2302375b4506dfc7b5 2026-03-30 00:05:02,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:05:02,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71b2302375b4506dfc7b5 HTTP/1.1" 200 None 2026-03-30 00:05:02,926 - lava_callback - DEBUG - Processing node 69c71b2302375b4506dfc7b5: baseline-arm64-clabbe 2026-03-30 00:05:02,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71b2302375b4506dfc7b5 2026-03-30 00:05:02,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:05:02,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71b2302375b4506dfc7b5 HTTP/1.1" 200 None 2026-03-30 00:05:02,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:05:02,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-30 00:05:02,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:05:02,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71b2302375b4506dfc7b5 HTTP/1.1" 200 None 2026-03-30 00:05:02,996 - lava_callback - ERROR - Error processing callback for node 69c71b2302375b4506dfc7b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54110 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:06:22,995 - lava_callback - INFO - Received callback for node 69c71b7502375b4506dfc7c1 INFO: 172.20.0.1:39894 - "POST /node/69c71b7502375b4506dfc7c1 HTTP/1.0" 202 Accepted 2026-03-30 00:06:23,000 - lava_callback - INFO - Starting async job submit for node 69c71b7502375b4506dfc7c1 2026-03-30 00:06:23,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:06:23,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71b7502375b4506dfc7c1 HTTP/1.1" 200 None 2026-03-30 00:06:23,016 - lava_callback - DEBUG - Processing node 69c71b7502375b4506dfc7c1: baseline-arm64-clabbe 2026-03-30 00:06:23,016 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71b7502375b4506dfc7c1 2026-03-30 00:06:23,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:06:23,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71b7502375b4506dfc7c1 HTTP/1.1" 200 None 2026-03-30 00:06:23,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:06:23,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-30 00:06:23,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:06:23,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71b7502375b4506dfc7c1 HTTP/1.1" 200 None 2026-03-30 00:06:23,085 - lava_callback - ERROR - Error processing callback for node 69c71b7502375b4506dfc7c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43098 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:07:43,100 - lava_callback - INFO - Received callback for node 69c71bb902375b4506dfc7c9 INFO: 172.20.0.1:44216 - "POST /node/69c71bb902375b4506dfc7c9 HTTP/1.0" 202 Accepted 2026-03-30 00:07:43,106 - lava_callback - INFO - Starting async job submit for node 69c71bb902375b4506dfc7c9 2026-03-30 00:07:43,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:07:43,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71bb902375b4506dfc7c9 HTTP/1.1" 200 None 2026-03-30 00:07:43,122 - lava_callback - DEBUG - Processing node 69c71bb902375b4506dfc7c9: baseline-arm-clabbe 2026-03-30 00:07:43,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71bb902375b4506dfc7c9 2026-03-30 00:07:43,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:07:43,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71bb902375b4506dfc7c9 HTTP/1.1" 200 None 2026-03-30 00:07:43,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:07:43,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-30 00:07:43,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:07:43,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71bb902375b4506dfc7c9 HTTP/1.1" 200 None 2026-03-30 00:07:43,185 - lava_callback - ERROR - Error processing callback for node 69c71bb902375b4506dfc7c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:08:03,038 - lava_callback - INFO - Received callback for node 69c71bdf02375b4506dfc7d9 INFO: 172.20.0.1:42494 - "POST /node/69c71bdf02375b4506dfc7d9 HTTP/1.0" 202 Accepted 2026-03-30 00:08:03,043 - lava_callback - INFO - Starting async job submit for node 69c71bdf02375b4506dfc7d9 2026-03-30 00:08:03,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:03,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71bdf02375b4506dfc7d9 HTTP/1.1" 200 None 2026-03-30 00:08:03,068 - lava_callback - DEBUG - Processing node 69c71bdf02375b4506dfc7d9: baseline-arm-clabbe 2026-03-30 00:08:03,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71bdf02375b4506dfc7d9 2026-03-30 00:08:03,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:03,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71bdf02375b4506dfc7d9 HTTP/1.1" 200 None 2026-03-30 00:08:03,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:03,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-30 00:08:03,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:03,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71bdf02375b4506dfc7d9 HTTP/1.1" 200 None 2026-03-30 00:08:03,121 - lava_callback - ERROR - Error processing callback for node 69c71bdf02375b4506dfc7d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51552 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:08:43,062 - lava_callback - INFO - Received callback for node 69c71c0502375b4506dfc7ec INFO: 172.20.0.1:43818 - "POST /node/69c71c0502375b4506dfc7ec HTTP/1.0" 202 Accepted 2026-03-30 00:08:43,067 - lava_callback - INFO - Starting async job submit for node 69c71c0502375b4506dfc7ec 2026-03-30 00:08:43,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:43,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71c0502375b4506dfc7ec HTTP/1.1" 200 None 2026-03-30 00:08:43,087 - lava_callback - DEBUG - Processing node 69c71c0502375b4506dfc7ec: baseline-arm-clabbe 2026-03-30 00:08:43,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71c0502375b4506dfc7ec 2026-03-30 00:08:43,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:43,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71c0502375b4506dfc7ec HTTP/1.1" 200 None 2026-03-30 00:08:43,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:43,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-30 00:08:43,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:08:43,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71c0502375b4506dfc7ec HTTP/1.1" 200 None 2026-03-30 00:08:43,149 - lava_callback - ERROR - Error processing callback for node 69c71c0502375b4506dfc7ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET / HTTP/1.0" 200 OK 2026-03-30 00:09:23,046 - lava_callback - INFO - Received callback for node 69c71c2b02375b4506dfc7f7 INFO: 172.20.0.1:36104 - "POST /node/69c71c2b02375b4506dfc7f7 HTTP/1.0" 202 Accepted 2026-03-30 00:09:23,051 - lava_callback - INFO - Starting async job submit for node 69c71c2b02375b4506dfc7f7 2026-03-30 00:09:23,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:09:23,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71c2b02375b4506dfc7f7 HTTP/1.1" 200 None 2026-03-30 00:09:23,068 - lava_callback - DEBUG - Processing node 69c71c2b02375b4506dfc7f7: baseline-arm-clabbe 2026-03-30 00:09:23,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c71c2b02375b4506dfc7f7 2026-03-30 00:09:23,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:09:23,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71c2b02375b4506dfc7f7 HTTP/1.1" 200 None 2026-03-30 00:09:23,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:09:23,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-30 00:09:23,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-30 00:09:23,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c71c2b02375b4506dfc7f7 HTTP/1.1" 200 None 2026-03-30 00:09:23,153 - lava_callback - ERROR - Error processing callback for node 69c71c2b02375b4506dfc7f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]