INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53952 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:31:55,218 - lava_callback - INFO - Received callback for node 69b0b7889d1917581825201c 2026-03-11 00:31:55,224 - lava_callback - INFO - Starting async job submit for node 69b0b7889d1917581825201c INFO: 172.21.0.1:36174 - "POST /node/69b0b7889d1917581825201c HTTP/1.0" 202 Accepted 2026-03-11 00:31:55,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:31:55,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b7889d1917581825201c HTTP/1.1" 200 None 2026-03-11 00:31:55,282 - lava_callback - DEBUG - Processing node 69b0b7889d1917581825201c: baseline-arm 2026-03-11 00:31:55,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:31:55,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:31:55,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b7889d1917581825201c/log.txt.gz 2026-03-11 00:31:55,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:31:56,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:31:56,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b7889d1917581825201c/lava-logs.yaml 2026-03-11 00:31:56,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:31:56,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:31:56,306 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b7889d1917581825201c/lava_callback.json.gz 2026-03-11 00:31:56,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:31:56,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b7889d1917581825201c HTTP/1.1" 200 None 2026-03-11 00:31:56,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:31:56,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5059d19175818251fdc HTTP/1.1" 200 None 2026-03-11 00:31:56,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:31:56,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b7889d1917581825201c HTTP/1.1" 200 None 2026-03-11 00:31:56,390 - lava_callback - INFO - Completed processing callback for node 69b0b7889d1917581825201c INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:32:26,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:32:26,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39762 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:34:28,543 - lava_callback - INFO - Received callback for node 69afdc68ae557f2600f0f865 INFO: 172.21.0.1:39772 - "POST /node/69afdc68ae557f2600f0f865 HTTP/1.0" 202 Accepted 2026-03-11 00:34:28,548 - lava_callback - INFO - Starting async job submit for node 69afdc68ae557f2600f0f865 2026-03-11 00:34:28,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:34:32,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdc68ae557f2600f0f865 HTTP/1.1" 200 None 2026-03-11 00:34:32,581 - lava_callback - DEBUG - Processing node 69afdc68ae557f2600f0f865: baseline-x86 2026-03-11 00:34:32,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:34:33,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:34:33,050 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69afdc68ae557f2600f0f865/log.txt.gz 2026-03-11 00:34:33,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:34:33,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:34:33,454 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69afdc68ae557f2600f0f865/lava-logs.yaml 2026-03-11 00:34:33,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:34:33,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:34:33,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69afdc68ae557f2600f0f865/lava_callback.json.gz 2026-03-11 00:34:33,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:34:33,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdc68ae557f2600f0f865 HTTP/1.1" 200 None 2026-03-11 00:34:33,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:34:33,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd852ae557f2600f0f05e HTTP/1.1" 200 None 2026-03-11 00:34:33,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:34:33,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdc68ae557f2600f0f865 HTTP/1.1" 200 None 2026-03-11 00:34:33,708 - lava_callback - INFO - Completed processing callback for node 69afdc68ae557f2600f0f865 INFO: 172.21.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54334 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:34:56,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:34:56,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:39:24,465 - lava_callback - INFO - Received callback for node 69b039a9ae557f2600f10dc7 INFO: 172.21.0.1:45074 - "POST /node/69b039a9ae557f2600f10dc7 HTTP/1.0" 202 Accepted 2026-03-11 00:39:24,474 - lava_callback - INFO - Starting async job submit for node 69b039a9ae557f2600f10dc7 2026-03-11 00:39:24,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:24,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b039a9ae557f2600f10dc7 HTTP/1.1" 200 None 2026-03-11 00:39:24,537 - lava_callback - DEBUG - Processing node 69b039a9ae557f2600f10dc7: baseline-x86 INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:39:25,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:39:25,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:39:25,518 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69b039a9ae557f2600f10dc7/log.txt.gz 2026-03-11 00:39:25,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:39:26,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:39:26,344 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69b039a9ae557f2600f10dc7/lava-logs.yaml 2026-03-11 00:39:26,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:39:26,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:39:26,771 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69b039a9ae557f2600f10dc7/lava_callback.json.gz 2026-03-11 00:39:26,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:26,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b039a9ae557f2600f10dc7 HTTP/1.1" 200 None 2026-03-11 00:39:26,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:26,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd852ae557f2600f0f05e HTTP/1.1" 200 None 2026-03-11 00:39:26,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:26,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b039a9ae557f2600f10dc7 HTTP/1.1" 200 None 2026-03-11 00:39:26,927 - lava_callback - INFO - Completed processing callback for node 69b039a9ae557f2600f10dc7 INFO: 172.21.0.1:32836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44130 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:39:49,100 - lava_callback - INFO - Received callback for node 69ae16c68ef4424e487b3382 INFO: 172.21.0.1:44146 - "POST /node/69ae16c68ef4424e487b3382 HTTP/1.0" 202 Accepted 2026-03-11 00:39:49,109 - lava_callback - INFO - Starting async job submit for node 69ae16c68ef4424e487b3382 2026-03-11 00:39:49,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:49,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae16c68ef4424e487b3382 HTTP/1.1" 200 None 2026-03-11 00:39:49,149 - lava_callback - DEBUG - Processing node 69ae16c68ef4424e487b3382: baseline-arm-baylibre 2026-03-11 00:39:49,150 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae16c68ef4424e487b3382 2026-03-11 00:39:49,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:49,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae16c68ef4424e487b3382 HTTP/1.1" 200 None 2026-03-11 00:39:49,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:49,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-11 00:39:49,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:49,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae16c68ef4424e487b3382 HTTP/1.1" 200 None 2026-03-11 00:39:49,248 - lava_callback - ERROR - Error processing callback for node 69ae16c68ef4424e487b3382: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:39:56,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:39:56,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42932 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:40:19,325 - lava_callback - INFO - Received callback for node 69b0b9d79d191758182523e6 INFO: 172.21.0.1:42942 - "POST /node/69b0b9d79d191758182523e6 HTTP/1.0" 202 Accepted 2026-03-11 00:40:19,332 - lava_callback - INFO - Starting async job submit for node 69b0b9d79d191758182523e6 2026-03-11 00:40:19,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:19,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d79d191758182523e6 HTTP/1.1" 200 None 2026-03-11 00:40:19,365 - lava_callback - DEBUG - Processing node 69b0b9d79d191758182523e6: baseline-arm 2026-03-11 00:40:19,370 - lava_callback - WARNING - Failed to upload log 2026-03-11 00:40:19,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:19,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:19,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d79d191758182523e6/lava-logs.yaml 2026-03-11 00:40:19,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:19,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:19,621 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d79d191758182523e6/lava_callback.json.gz 2026-03-11 00:40:19,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:19,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d79d191758182523e6 HTTP/1.1" 200 None 2026-03-11 00:40:19,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:19,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:40:19,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:19,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9d79d191758182523e6 HTTP/1.1" 200 None 2026-03-11 00:40:19,683 - lava_callback - INFO - Completed processing callback for node 69b0b9d79d191758182523e6 INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:40:26,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:26,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:40:40,033 - lava_callback - INFO - Received callback for node 69b0b9d89d191758182523e9 INFO: 172.21.0.1:41944 - "POST /node/69b0b9d89d191758182523e9 HTTP/1.0" 202 Accepted 2026-03-11 00:40:40,082 - lava_callback - INFO - Starting async job submit for node 69b0b9d89d191758182523e9 2026-03-11 00:40:40,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:40,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d89d191758182523e9 HTTP/1.1" 200 None 2026-03-11 00:40:40,123 - lava_callback - DEBUG - Processing node 69b0b9d89d191758182523e9: baseline-arm 2026-03-11 00:40:40,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:40,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:40,372 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d89d191758182523e9/log.txt.gz 2026-03-11 00:40:40,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:40,705 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:40,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d89d191758182523e9/lava-logs.yaml 2026-03-11 00:40:40,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:40,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:40,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d89d191758182523e9/lava_callback.json.gz 2026-03-11 00:40:40,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:40,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d89d191758182523e9 HTTP/1.1" 200 None 2026-03-11 00:40:40,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:40,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:40:40,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:40,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9d89d191758182523e9 HTTP/1.1" 200 None 2026-03-11 00:40:40,910 - lava_callback - INFO - Completed processing callback for node 69b0b9d89d191758182523e9 INFO: 172.21.0.1:41948 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:40:47,117 - lava_callback - INFO - Received callback for node 69b0b9db9d191758182523f9 INFO: 172.21.0.1:41954 - "POST /node/69b0b9db9d191758182523f9 HTTP/1.0" 202 Accepted 2026-03-11 00:40:47,169 - lava_callback - INFO - Starting async job submit for node 69b0b9db9d191758182523f9 2026-03-11 00:40:47,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:47,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9db9d191758182523f9 HTTP/1.1" 200 None 2026-03-11 00:40:47,224 - lava_callback - DEBUG - Processing node 69b0b9db9d191758182523f9: baseline-arm 2026-03-11 00:40:47,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:47,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:47,526 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9db9d191758182523f9/log.txt.gz 2026-03-11 00:40:47,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:47,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:47,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9db9d191758182523f9/lava-logs.yaml 2026-03-11 00:40:47,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:40:47,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:40:47,935 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9db9d191758182523f9/lava_callback.json.gz 2026-03-11 00:40:47,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:47,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9db9d191758182523f9 HTTP/1.1" 200 None 2026-03-11 00:40:47,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:47,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:40:47,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:40:48,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9db9d191758182523f9 HTTP/1.1" 200 None 2026-03-11 00:40:48,017 - lava_callback - INFO - Completed processing callback for node 69b0b9db9d191758182523f9 2026-03-11 00:40:56,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:37016 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:40:56,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60950 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:41:13,806 - lava_callback - INFO - Received callback for node 69b0b9da9d191758182523ef INFO: 172.21.0.1:43484 - "POST /node/69b0b9da9d191758182523ef HTTP/1.0" 202 Accepted 2026-03-11 00:41:13,817 - lava_callback - INFO - Starting async job submit for node 69b0b9da9d191758182523ef 2026-03-11 00:41:13,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:13,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9da9d191758182523ef HTTP/1.1" 200 None 2026-03-11 00:41:13,888 - lava_callback - DEBUG - Processing node 69b0b9da9d191758182523ef: baseline-arm 2026-03-11 00:41:14,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:14,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:14,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9da9d191758182523ef/log.txt.gz 2026-03-11 00:41:14,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:43486 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-11 00:41:15,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:15,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9da9d191758182523ef/lava-logs.yaml 2026-03-11 00:41:15,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:15,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:15,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9da9d191758182523ef/lava_callback.json.gz 2026-03-11 00:41:15,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:15,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9da9d191758182523ef HTTP/1.1" 200 None 2026-03-11 00:41:15,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:15,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:15,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:15,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9da9d191758182523ef HTTP/1.1" 200 None 2026-03-11 00:41:15,738 - lava_callback - INFO - Completed processing callback for node 69b0b9da9d191758182523ef INFO: 172.21.0.1:43488 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:41:18,134 - lava_callback - INFO - Received callback for node 69b0b9d79d191758182523e3 INFO: 172.21.0.1:43490 - "POST /node/69b0b9d79d191758182523e3 HTTP/1.0" 202 Accepted 2026-03-11 00:41:18,143 - lava_callback - INFO - Starting async job submit for node 69b0b9d79d191758182523e3 2026-03-11 00:41:18,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:18,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d79d191758182523e3 HTTP/1.1" 200 None 2026-03-11 00:41:18,211 - lava_callback - DEBUG - Processing node 69b0b9d79d191758182523e3: baseline-arm 2026-03-11 00:41:18,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:18,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:18,975 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d79d191758182523e3/log.txt.gz 2026-03-11 00:41:19,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:19,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:19,961 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d79d191758182523e3/lava-logs.yaml 2026-03-11 00:41:19,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:20,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:20,339 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d79d191758182523e3/lava_callback.json.gz 2026-03-11 00:41:20,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:20,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d79d191758182523e3 HTTP/1.1" 200 None 2026-03-11 00:41:20,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:20,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:20,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:20,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9d79d191758182523e3 HTTP/1.1" 200 None 2026-03-11 00:41:20,478 - lava_callback - INFO - Completed processing callback for node 69b0b9d79d191758182523e3 2026-03-11 00:41:21,143 - lava_callback - INFO - Received callback for node 69b0b9fa9d19175818252486 INFO: 172.21.0.1:59842 - "POST /node/69b0b9fa9d19175818252486 HTTP/1.0" 202 Accepted 2026-03-11 00:41:21,193 - lava_callback - INFO - Starting async job submit for node 69b0b9fa9d19175818252486 2026-03-11 00:41:21,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:21,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9fa9d19175818252486 HTTP/1.1" 200 None 2026-03-11 00:41:21,254 - lava_callback - DEBUG - Processing node 69b0b9fa9d19175818252486: baseline-arm-pengutronix 2026-03-11 00:41:21,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:21,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:21,701 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9fa9d19175818252486/log.txt.gz 2026-03-11 00:41:21,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:22,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:22,137 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9fa9d19175818252486/lava-logs.yaml 2026-03-11 00:41:22,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:22,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:22,518 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9fa9d19175818252486/lava_callback.json.gz 2026-03-11 00:41:22,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:22,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9fa9d19175818252486 HTTP/1.1" 200 None 2026-03-11 00:41:22,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:22,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:22,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:22,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9fa9d19175818252486 HTTP/1.1" 200 None 2026-03-11 00:41:22,663 - lava_callback - INFO - Completed processing callback for node 69b0b9fa9d19175818252486 INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:41:25,472 - lava_callback - INFO - Received callback for node 69b0b9f79d19175818252452 INFO: 172.21.0.1:59860 - "POST /node/69b0b9f79d19175818252452 HTTP/1.0" 202 Accepted 2026-03-11 00:41:25,483 - lava_callback - INFO - Starting async job submit for node 69b0b9f79d19175818252452 2026-03-11 00:41:25,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:25,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f79d19175818252452 HTTP/1.1" 200 None 2026-03-11 00:41:25,547 - lava_callback - DEBUG - Processing node 69b0b9f79d19175818252452: baseline-arm-pengutronix 2026-03-11 00:41:25,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:26,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:26,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f79d19175818252452/log.txt.gz 2026-03-11 00:41:26,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59876 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:41:26,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:26,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:41:26,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:26,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f79d19175818252452/lava-logs.yaml 2026-03-11 00:41:26,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:27,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:27,095 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f79d19175818252452/lava_callback.json.gz 2026-03-11 00:41:27,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:27,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f79d19175818252452 HTTP/1.1" 200 None 2026-03-11 00:41:27,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:27,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:27,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:27,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9f79d19175818252452 HTTP/1.1" 200 None 2026-03-11 00:41:27,237 - lava_callback - INFO - Completed processing callback for node 69b0b9f79d19175818252452 2026-03-11 00:41:29,162 - lava_callback - INFO - Received callback for node 69ae172a8ef4424e487b33cf INFO: 172.21.0.1:59892 - "POST /node/69ae172a8ef4424e487b33cf HTTP/1.0" 202 Accepted 2026-03-11 00:41:29,173 - lava_callback - INFO - Starting async job submit for node 69ae172a8ef4424e487b33cf 2026-03-11 00:41:29,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:29,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae172a8ef4424e487b33cf HTTP/1.1" 200 None 2026-03-11 00:41:29,200 - lava_callback - DEBUG - Processing node 69ae172a8ef4424e487b33cf: baseline-arm-baylibre 2026-03-11 00:41:29,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae172a8ef4424e487b33cf 2026-03-11 00:41:29,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:29,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae172a8ef4424e487b33cf HTTP/1.1" 200 None 2026-03-11 00:41:29,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:29,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-11 00:41:29,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:29,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae172a8ef4424e487b33cf HTTP/1.1" 200 None 2026-03-11 00:41:29,327 - lava_callback - ERROR - Error processing callback for node 69ae172a8ef4424e487b33cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:41:35,317 - lava_callback - INFO - Received callback for node 69b0b9e49d1917581825240d INFO: 172.21.0.1:56162 - "POST /node/69b0b9e49d1917581825240d HTTP/1.0" 202 Accepted 2026-03-11 00:41:35,326 - lava_callback - INFO - Starting async job submit for node 69b0b9e49d1917581825240d 2026-03-11 00:41:35,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:35,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e49d1917581825240d HTTP/1.1" 200 None 2026-03-11 00:41:35,394 - lava_callback - DEBUG - Processing node 69b0b9e49d1917581825240d: baseline-arm-broonie 2026-03-11 00:41:35,415 - lava_callback - INFO - Received callback for node 69b0b9eb9d19175818252428 2026-03-11 00:41:35,431 - lava_callback - INFO - Starting async job submit for node 69b0b9eb9d19175818252428 INFO: 172.21.0.1:56178 - "POST /node/69b0b9eb9d19175818252428 HTTP/1.0" 202 Accepted 2026-03-11 00:41:35,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:35,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9eb9d19175818252428 HTTP/1.1" 200 None 2026-03-11 00:41:35,566 - lava_callback - DEBUG - Processing node 69b0b9eb9d19175818252428: baseline-arm-clabbe 2026-03-11 00:41:36,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:36,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:36,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:36,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e49d1917581825240d/log.txt.gz INFO: 172.21.0.1:56194 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:41:36,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:36,596 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9eb9d19175818252428/log.txt.gz 2026-03-11 00:41:36,805 - lava_callback - INFO - Received callback for node 69b0b9e59d19175818252414 2026-03-11 00:41:36,819 - lava_callback - INFO - Starting async job submit for node 69b0b9e59d19175818252414 INFO: 172.21.0.1:56210 - "POST /node/69b0b9e59d19175818252414 HTTP/1.0" 202 Accepted 2026-03-11 00:41:36,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:36,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:36,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e59d19175818252414 HTTP/1.1" 200 None 2026-03-11 00:41:36,980 - lava_callback - DEBUG - Processing node 69b0b9e59d19175818252414: baseline-arm-broonie 2026-03-11 00:41:37,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:37,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:37,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:37,573 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e49d1917581825240d/lava-logs.yaml 2026-03-11 00:41:37,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:37,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:37,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9eb9d19175818252428/lava-logs.yaml 2026-03-11 00:41:37,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:37,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:37,797 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e59d19175818252414/log.txt.gz 2026-03-11 00:41:38,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:38,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:38,057 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e49d1917581825240d/lava_callback.json.gz 2026-03-11 00:41:38,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:38,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e49d1917581825240d HTTP/1.1" 200 None 2026-03-11 00:41:38,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:38,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:38,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:38,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9e49d1917581825240d HTTP/1.1" 200 None 2026-03-11 00:41:38,203 - lava_callback - INFO - Completed processing callback for node 69b0b9e49d1917581825240d 2026-03-11 00:41:38,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:38,324 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9eb9d19175818252428/lava_callback.json.gz 2026-03-11 00:41:38,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:38,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9eb9d19175818252428 HTTP/1.1" 200 None 2026-03-11 00:41:38,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:38,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:38,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:38,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9eb9d19175818252428 HTTP/1.1" 200 None 2026-03-11 00:41:38,509 - lava_callback - INFO - Completed processing callback for node 69b0b9eb9d19175818252428 2026-03-11 00:41:38,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:38,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e59d19175818252414/lava-logs.yaml 2026-03-11 00:41:38,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:39,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:39,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e59d19175818252414/lava_callback.json.gz 2026-03-11 00:41:39,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:39,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e59d19175818252414 HTTP/1.1" 200 None 2026-03-11 00:41:39,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:39,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:39,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:39,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9e59d19175818252414 HTTP/1.1" 200 None 2026-03-11 00:41:39,469 - lava_callback - INFO - Completed processing callback for node 69b0b9e59d19175818252414 2026-03-11 00:41:40,609 - lava_callback - INFO - Received callback for node 69b0b9e79d19175818252418 INFO: 172.21.0.1:33670 - "POST /node/69b0b9e79d19175818252418 HTTP/1.0" 202 Accepted 2026-03-11 00:41:40,619 - lava_callback - INFO - Starting async job submit for node 69b0b9e79d19175818252418 2026-03-11 00:41:40,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:40,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e79d19175818252418 HTTP/1.1" 200 None 2026-03-11 00:41:40,734 - lava_callback - DEBUG - Processing node 69b0b9e79d19175818252418: baseline-arm-broonie 2026-03-11 00:41:41,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:41,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:41,538 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e79d19175818252418/log.txt.gz 2026-03-11 00:41:41,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:42,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:42,305 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e79d19175818252418/lava-logs.yaml 2026-03-11 00:41:42,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:41:42,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:41:42,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e79d19175818252418/lava_callback.json.gz 2026-03-11 00:41:42,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:42,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e79d19175818252418 HTTP/1.1" 200 None 2026-03-11 00:41:42,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:42,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:41:42,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:42,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9e79d19175818252418 HTTP/1.1" 200 None 2026-03-11 00:41:42,933 - lava_callback - INFO - Completed processing callback for node 69b0b9e79d19175818252418 INFO: 172.21.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60954 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:41:56,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:41:56,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-11 00:42:02,060 - lava_callback - INFO - Received callback for node 69b0b9f39d19175818252443 INFO: 172.21.0.1:37792 - "POST /node/69b0b9f39d19175818252443 HTTP/1.0" 202 Accepted 2026-03-11 00:42:02,072 - lava_callback - INFO - Starting async job submit for node 69b0b9f39d19175818252443 2026-03-11 00:42:02,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:02,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f39d19175818252443 HTTP/1.1" 200 None 2026-03-11 00:42:02,147 - lava_callback - DEBUG - Processing node 69b0b9f39d19175818252443: baseline-arm-kontron 2026-03-11 00:42:02,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:02,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:02,879 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b0b9f39d19175818252443/log.txt.gz 2026-03-11 00:42:03,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:03,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:03,550 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b0b9f39d19175818252443/lava-logs.yaml 2026-03-11 00:42:03,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:04,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:04,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b0b9f39d19175818252443/lava_callback.json.gz 2026-03-11 00:42:04,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:04,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f39d19175818252443 HTTP/1.1" 200 None 2026-03-11 00:42:04,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:04,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:42:04,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:04,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9f39d19175818252443 HTTP/1.1" 200 None 2026-03-11 00:42:04,231 - lava_callback - INFO - Completed processing callback for node 69b0b9f39d19175818252443 INFO: 172.21.0.1:37808 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:42:09,153 - lava_callback - INFO - Received callback for node 69ae17568ef4424e487b33ef INFO: 172.21.0.1:37824 - "POST /node/69ae17568ef4424e487b33ef HTTP/1.0" 202 Accepted 2026-03-11 00:42:09,162 - lava_callback - INFO - Starting async job submit for node 69ae17568ef4424e487b33ef 2026-03-11 00:42:09,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:09,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17568ef4424e487b33ef HTTP/1.1" 200 None 2026-03-11 00:42:09,208 - lava_callback - DEBUG - Processing node 69ae17568ef4424e487b33ef: baseline-arm-baylibre 2026-03-11 00:42:09,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae17568ef4424e487b33ef 2026-03-11 00:42:09,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:09,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17568ef4424e487b33ef HTTP/1.1" 200 None 2026-03-11 00:42:09,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:09,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-11 00:42:09,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:09,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17568ef4424e487b33ef HTTP/1.1" 200 None 2026-03-11 00:42:09,315 - lava_callback - ERROR - Error processing callback for node 69ae17568ef4424e487b33ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46390 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-11 00:42:15,199 - lava_callback - INFO - Received callback for node 69b0b9e39d1917581825240a INFO: 172.21.0.1:46400 - "POST /node/69b0b9e39d1917581825240a HTTP/1.0" 202 Accepted 2026-03-11 00:42:15,205 - lava_callback - INFO - Starting async job submit for node 69b0b9e39d1917581825240a 2026-03-11 00:42:15,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:15,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e39d1917581825240a HTTP/1.1" 200 None 2026-03-11 00:42:15,249 - lava_callback - DEBUG - Processing node 69b0b9e39d1917581825240a: baseline-arm-broonie 2026-03-11 00:42:15,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:15,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:15,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e39d1917581825240a/log.txt.gz 2026-03-11 00:42:15,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:16,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:16,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e39d1917581825240a/lava-logs.yaml 2026-03-11 00:42:16,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46412 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:42:16,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:16,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b0b9e39d1917581825240a/lava_callback.json.gz 2026-03-11 00:42:16,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:16,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9e39d1917581825240a HTTP/1.1" 200 None 2026-03-11 00:42:16,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:16,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:42:16,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:16,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9e39d1917581825240a HTTP/1.1" 200 None 2026-03-11 00:42:16,963 - lava_callback - INFO - Completed processing callback for node 69b0b9e39d1917581825240a INFO: 172.21.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:42:26,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:26,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:42:27,401 - lava_callback - INFO - Received callback for node 69b0ba119d191758182524a6 INFO: 172.21.0.1:51268 - "POST /node/69b0ba119d191758182524a6 HTTP/1.0" 202 Accepted 2026-03-11 00:42:27,408 - lava_callback - INFO - Starting async job submit for node 69b0ba119d191758182524a6 2026-03-11 00:42:27,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:27,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba119d191758182524a6 HTTP/1.1" 200 None 2026-03-11 00:42:27,446 - lava_callback - DEBUG - Processing node 69b0ba119d191758182524a6: ltp-dio 2026-03-11 00:42:27,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:27,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:27,975 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69b0ba119d191758182524a6/log.txt.gz 2026-03-11 00:42:28,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:28,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:28,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69b0ba119d191758182524a6/lava-logs.yaml 2026-03-11 00:42:28,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:28,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:28,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69b0ba119d191758182524a6/lava_callback.json.gz 2026-03-11 00:42:28,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:28,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba119d191758182524a6 HTTP/1.1" 200 None 2026-03-11 00:42:28,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:28,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:42:28,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:28,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba119d191758182524a6 HTTP/1.1" 200 None 2026-03-11 00:42:28,615 - lava_callback - INFO - Completed processing callback for node 69b0ba119d191758182524a6 2026-03-11 00:42:29,163 - lava_callback - INFO - Received callback for node 69ae17698ef4424e487b3433 INFO: 172.21.0.1:51278 - "POST /node/69ae17698ef4424e487b3433 HTTP/1.0" 202 Accepted 2026-03-11 00:42:29,169 - lava_callback - INFO - Starting async job submit for node 69ae17698ef4424e487b3433 2026-03-11 00:42:29,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:29,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17698ef4424e487b3433 HTTP/1.1" 200 None 2026-03-11 00:42:29,188 - lava_callback - DEBUG - Processing node 69ae17698ef4424e487b3433: baseline-arm-baylibre 2026-03-11 00:42:29,188 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae17698ef4424e487b3433 2026-03-11 00:42:29,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:29,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17698ef4424e487b3433 HTTP/1.1" 200 None 2026-03-11 00:42:29,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:29,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:29,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:29,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17698ef4424e487b3433 HTTP/1.1" 200 None 2026-03-11 00:42:29,238 - lava_callback - ERROR - Error processing callback for node 69ae17698ef4424e487b3433: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:29,990 - lava_callback - INFO - Received callback for node 69ae176a8ef4424e487b3436 INFO: 172.21.0.1:45954 - "POST /node/69ae176a8ef4424e487b3436 HTTP/1.0" 202 Accepted 2026-03-11 00:42:29,995 - lava_callback - INFO - Starting async job submit for node 69ae176a8ef4424e487b3436 2026-03-11 00:42:29,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae176a8ef4424e487b3436 HTTP/1.1" 200 None 2026-03-11 00:42:30,013 - lava_callback - DEBUG - Processing node 69ae176a8ef4424e487b3436: baseline-arm-baylibre 2026-03-11 00:42:30,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae176a8ef4424e487b3436 2026-03-11 00:42:30,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae176a8ef4424e487b3436 HTTP/1.1" 200 None 2026-03-11 00:42:30,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:30,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae176a8ef4424e487b3436 HTTP/1.1" 200 None 2026-03-11 00:42:30,094 - lava_callback - ERROR - Error processing callback for node 69ae176a8ef4424e487b3436: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:30,660 - lava_callback - INFO - Received callback for node 69ae176c8ef4424e487b3439 INFO: 172.21.0.1:45968 - "POST /node/69ae176c8ef4424e487b3439 HTTP/1.0" 202 Accepted 2026-03-11 00:42:30,665 - lava_callback - INFO - Starting async job submit for node 69ae176c8ef4424e487b3439 2026-03-11 00:42:30,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae176c8ef4424e487b3439 HTTP/1.1" 200 None 2026-03-11 00:42:30,682 - lava_callback - DEBUG - Processing node 69ae176c8ef4424e487b3439: baseline-arm-baylibre 2026-03-11 00:42:30,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae176c8ef4424e487b3439 2026-03-11 00:42:30,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae176c8ef4424e487b3439 HTTP/1.1" 200 None 2026-03-11 00:42:30,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:30,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:30,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae176c8ef4424e487b3439 HTTP/1.1" 200 None 2026-03-11 00:42:30,761 - lava_callback - ERROR - Error processing callback for node 69ae176c8ef4424e487b3439: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:31,330 - lava_callback - INFO - Received callback for node 69ae176d8ef4424e487b343c INFO: 172.21.0.1:45976 - "POST /node/69ae176d8ef4424e487b343c HTTP/1.0" 202 Accepted 2026-03-11 00:42:31,336 - lava_callback - INFO - Starting async job submit for node 69ae176d8ef4424e487b343c 2026-03-11 00:42:31,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:31,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae176d8ef4424e487b343c HTTP/1.1" 200 None 2026-03-11 00:42:31,352 - lava_callback - DEBUG - Processing node 69ae176d8ef4424e487b343c: baseline-arm-baylibre 2026-03-11 00:42:31,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae176d8ef4424e487b343c 2026-03-11 00:42:31,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:31,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae176d8ef4424e487b343c HTTP/1.1" 200 None 2026-03-11 00:42:31,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:31,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:31,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:31,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae176d8ef4424e487b343c HTTP/1.1" 200 None 2026-03-11 00:42:31,411 - lava_callback - ERROR - Error processing callback for node 69ae176d8ef4424e487b343c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:32,031 - lava_callback - INFO - Received callback for node 69ae17708ef4424e487b3447 INFO: 172.21.0.1:45986 - "POST /node/69ae17708ef4424e487b3447 HTTP/1.0" 202 Accepted 2026-03-11 00:42:32,040 - lava_callback - INFO - Starting async job submit for node 69ae17708ef4424e487b3447 2026-03-11 00:42:32,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17708ef4424e487b3447 HTTP/1.1" 200 None 2026-03-11 00:42:32,062 - lava_callback - DEBUG - Processing node 69ae17708ef4424e487b3447: baseline-arm-baylibre 2026-03-11 00:42:32,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae17708ef4424e487b3447 2026-03-11 00:42:32,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17708ef4424e487b3447 HTTP/1.1" 200 None 2026-03-11 00:42:32,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:32,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17708ef4424e487b3447 HTTP/1.1" 200 None 2026-03-11 00:42:32,125 - lava_callback - ERROR - Error processing callback for node 69ae17708ef4424e487b3447: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:32,694 - lava_callback - INFO - Received callback for node 69ae17718ef4424e487b344a INFO: 172.21.0.1:45994 - "POST /node/69ae17718ef4424e487b344a HTTP/1.0" 202 Accepted 2026-03-11 00:42:32,700 - lava_callback - INFO - Starting async job submit for node 69ae17718ef4424e487b344a 2026-03-11 00:42:32,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17718ef4424e487b344a HTTP/1.1" 200 None 2026-03-11 00:42:32,716 - lava_callback - DEBUG - Processing node 69ae17718ef4424e487b344a: baseline-arm-baylibre 2026-03-11 00:42:32,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae17718ef4424e487b344a 2026-03-11 00:42:32,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17718ef4424e487b344a HTTP/1.1" 200 None 2026-03-11 00:42:32,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:32,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17718ef4424e487b344a HTTP/1.1" 200 None 2026-03-11 00:42:32,777 - lava_callback - ERROR - Error processing callback for node 69ae17718ef4424e487b344a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:32,805 - lava_callback - INFO - Received callback for node 69b0b9f69d1917581825244e INFO: 172.21.0.1:45998 - "POST /node/69b0b9f69d1917581825244e HTTP/1.0" 202 Accepted 2026-03-11 00:42:32,823 - lava_callback - INFO - Starting async job submit for node 69b0b9f69d1917581825244e 2026-03-11 00:42:32,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:32,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f69d1917581825244e HTTP/1.1" 200 None 2026-03-11 00:42:32,893 - lava_callback - DEBUG - Processing node 69b0b9f69d1917581825244e: baseline-arm-pengutronix 2026-03-11 00:42:33,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:33,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:33,197 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f69d1917581825244e/log.txt.gz 2026-03-11 00:42:33,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:33,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:33,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f69d1917581825244e/lava-logs.yaml 2026-03-11 00:42:33,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:33,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:33,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f69d1917581825244e/lava_callback.json.gz 2026-03-11 00:42:33,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:33,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f69d1917581825244e HTTP/1.1" 200 None 2026-03-11 00:42:33,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:33,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:42:33,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:33,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9f69d1917581825244e HTTP/1.1" 200 None 2026-03-11 00:42:33,758 - lava_callback - INFO - Completed processing callback for node 69b0b9f69d1917581825244e 2026-03-11 00:42:34,153 - lava_callback - INFO - Received callback for node 69b08e6cae557f2600f11713 INFO: 172.21.0.1:46006 - "POST /node/69b08e6cae557f2600f11713 HTTP/1.0" 202 Accepted 2026-03-11 00:42:34,159 - lava_callback - INFO - Starting async job submit for node 69b08e6cae557f2600f11713 2026-03-11 00:42:34,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:34,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b08e6cae557f2600f11713 HTTP/1.1" 200 None 2026-03-11 00:42:34,202 - lava_callback - DEBUG - Processing node 69b08e6cae557f2600f11713: baseline-x86 2026-03-11 00:42:34,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:34,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:34,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69b08e6cae557f2600f11713/log.txt.gz 2026-03-11 00:42:34,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:35,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:35,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69b08e6cae557f2600f11713/lava-logs.yaml 2026-03-11 00:42:35,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:35,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:35,167 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69b08e6cae557f2600f11713/lava_callback.json.gz 2026-03-11 00:42:35,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:35,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b08e6cae557f2600f11713 HTTP/1.1" 200 None 2026-03-11 00:42:35,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:35,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd852ae557f2600f0f05e HTTP/1.1" 200 None 2026-03-11 00:42:35,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:35,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b08e6cae557f2600f11713 HTTP/1.1" 200 None 2026-03-11 00:42:35,250 - lava_callback - INFO - Completed processing callback for node 69b08e6cae557f2600f11713 INFO: 172.21.0.1:46008 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:42:41,103 - lava_callback - INFO - Received callback for node 69ae17788ef4424e487b3490 INFO: 172.21.0.1:36264 - "POST /node/69ae17788ef4424e487b3490 HTTP/1.0" 202 Accepted 2026-03-11 00:42:41,109 - lava_callback - INFO - Starting async job submit for node 69ae17788ef4424e487b3490 2026-03-11 00:42:41,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17788ef4424e487b3490 HTTP/1.1" 200 None 2026-03-11 00:42:41,125 - lava_callback - DEBUG - Processing node 69ae17788ef4424e487b3490: baseline-arm-clabbe 2026-03-11 00:42:41,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae17788ef4424e487b3490 2026-03-11 00:42:41,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17788ef4424e487b3490 HTTP/1.1" 200 None 2026-03-11 00:42:41,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:41,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17788ef4424e487b3490 HTTP/1.1" 200 None 2026-03-11 00:42:41,184 - lava_callback - ERROR - Error processing callback for node 69ae17788ef4424e487b3490: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:41,785 - lava_callback - INFO - Received callback for node 69ae177c8ef4424e487b34ab INFO: 172.21.0.1:36270 - "POST /node/69ae177c8ef4424e487b34ab HTTP/1.0" 202 Accepted 2026-03-11 00:42:41,790 - lava_callback - INFO - Starting async job submit for node 69ae177c8ef4424e487b34ab 2026-03-11 00:42:41,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177c8ef4424e487b34ab HTTP/1.1" 200 None 2026-03-11 00:42:41,807 - lava_callback - DEBUG - Processing node 69ae177c8ef4424e487b34ab: baseline-arm-clabbe 2026-03-11 00:42:41,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae177c8ef4424e487b34ab 2026-03-11 00:42:41,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177c8ef4424e487b34ab HTTP/1.1" 200 None 2026-03-11 00:42:41,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:41,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:41,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae177c8ef4424e487b34ab HTTP/1.1" 200 None 2026-03-11 00:42:41,864 - lava_callback - ERROR - Error processing callback for node 69ae177c8ef4424e487b34ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:42,471 - lava_callback - INFO - Received callback for node 69ae177d8ef4424e487b34af INFO: 172.21.0.1:36276 - "POST /node/69ae177d8ef4424e487b34af HTTP/1.0" 202 Accepted 2026-03-11 00:42:42,477 - lava_callback - INFO - Starting async job submit for node 69ae177d8ef4424e487b34af 2026-03-11 00:42:42,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:42,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177d8ef4424e487b34af HTTP/1.1" 200 None 2026-03-11 00:42:42,492 - lava_callback - DEBUG - Processing node 69ae177d8ef4424e487b34af: baseline-arm-clabbe 2026-03-11 00:42:42,493 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae177d8ef4424e487b34af 2026-03-11 00:42:42,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:42,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177d8ef4424e487b34af HTTP/1.1" 200 None 2026-03-11 00:42:42,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:42,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:42,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:42,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae177d8ef4424e487b34af HTTP/1.1" 200 None 2026-03-11 00:42:42,558 - lava_callback - ERROR - Error processing callback for node 69ae177d8ef4424e487b34af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:42:43,164 - lava_callback - INFO - Received callback for node 69ae177e8ef4424e487b34b3 INFO: 172.21.0.1:36290 - "POST /node/69ae177e8ef4424e487b34b3 HTTP/1.0" 202 Accepted 2026-03-11 00:42:43,169 - lava_callback - INFO - Starting async job submit for node 69ae177e8ef4424e487b34b3 2026-03-11 00:42:43,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:43,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177e8ef4424e487b34b3 HTTP/1.1" 200 None 2026-03-11 00:42:43,185 - lava_callback - DEBUG - Processing node 69ae177e8ef4424e487b34b3: baseline-arm-clabbe 2026-03-11 00:42:43,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae177e8ef4424e487b34b3 2026-03-11 00:42:43,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:43,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae177e8ef4424e487b34b3 HTTP/1.1" 200 None 2026-03-11 00:42:43,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:43,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:42:43,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:43,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae177e8ef4424e487b34b3 HTTP/1.1" 200 None 2026-03-11 00:42:43,236 - lava_callback - ERROR - Error processing callback for node 69ae177e8ef4424e487b34b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36302 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:42:52,150 - lava_callback - INFO - Received callback for node 69b0ba779d1917581825256b INFO: 172.21.0.1:46288 - "POST /node/69b0ba779d1917581825256b HTTP/1.0" 202 Accepted 2026-03-11 00:42:52,156 - lava_callback - INFO - Starting async job submit for node 69b0ba779d1917581825256b 2026-03-11 00:42:52,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:52,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba779d1917581825256b HTTP/1.1" 200 None 2026-03-11 00:42:52,174 - lava_callback - DEBUG - Processing node 69b0ba779d1917581825256b: baseline-arm 2026-03-11 00:42:52,179 - lava_callback - WARNING - Failed to upload log 2026-03-11 00:42:52,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:52,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:52,336 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0ba779d1917581825256b/lava-logs.yaml 2026-03-11 00:42:52,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:42:52,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:42:52,478 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0ba779d1917581825256b/lava_callback.json.gz 2026-03-11 00:42:52,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:52,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba779d1917581825256b HTTP/1.1" 200 None 2026-03-11 00:42:52,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:52,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:42:52,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:52,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba779d1917581825256b HTTP/1.1" 200 None 2026-03-11 00:42:52,544 - lava_callback - INFO - Completed processing callback for node 69b0ba779d1917581825256b INFO: 172.21.0.1:46294 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:42:56,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:42:56,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-11 00:43:00,927 - lava_callback - INFO - Received callback for node 69ae17818ef4424e487b34c1 INFO: 172.21.0.1:59680 - "POST /node/69ae17818ef4424e487b34c1 HTTP/1.0" 202 Accepted 2026-03-11 00:43:00,933 - lava_callback - INFO - Starting async job submit for node 69ae17818ef4424e487b34c1 2026-03-11 00:43:00,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:00,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17818ef4424e487b34c1 HTTP/1.1" 200 None 2026-03-11 00:43:00,950 - lava_callback - DEBUG - Processing node 69ae17818ef4424e487b34c1: baseline-arm-clabbe 2026-03-11 00:43:00,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae17818ef4424e487b34c1 2026-03-11 00:43:00,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:00,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17818ef4424e487b34c1 HTTP/1.1" 200 None 2026-03-11 00:43:00,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:00,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 00:43:00,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:01,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17818ef4424e487b34c1 HTTP/1.1" 200 None 2026-03-11 00:43:01,008 - lava_callback - ERROR - Error processing callback for node 69ae17818ef4424e487b34c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:43:02,461 - lava_callback - INFO - Received callback for node 69b0b9ec9d1917581825242c INFO: 172.21.0.1:59696 - "POST /node/69b0b9ec9d1917581825242c HTTP/1.0" 202 Accepted 2026-03-11 00:43:02,467 - lava_callback - INFO - Starting async job submit for node 69b0b9ec9d1917581825242c 2026-03-11 00:43:02,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:02,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ec9d1917581825242c HTTP/1.1" 200 None 2026-03-11 00:43:02,503 - lava_callback - DEBUG - Processing node 69b0b9ec9d1917581825242c: baseline-arm-clabbe 2026-03-11 00:43:02,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:43:02,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:43:02,818 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9ec9d1917581825242c/log.txt.gz 2026-03-11 00:43:02,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:43:03,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:43:03,121 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9ec9d1917581825242c/lava-logs.yaml 2026-03-11 00:43:03,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:43:03,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:43:03,255 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9ec9d1917581825242c/lava_callback.json.gz 2026-03-11 00:43:03,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:03,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ec9d1917581825242c HTTP/1.1" 200 None 2026-03-11 00:43:03,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:03,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:43:03,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:03,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ec9d1917581825242c HTTP/1.1" 200 None 2026-03-11 00:43:03,343 - lava_callback - INFO - Completed processing callback for node 69b0b9ec9d1917581825242c INFO: 172.21.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58760 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:43:26,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:43:26,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47248 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:44:09,354 - lava_callback - INFO - Received callback for node 69ae17d18ef4424e487b359e INFO: 172.21.0.1:47250 - "POST /node/69ae17d18ef4424e487b359e HTTP/1.0" 202 Accepted 2026-03-11 00:44:09,360 - lava_callback - INFO - Starting async job submit for node 69ae17d18ef4424e487b359e 2026-03-11 00:44:09,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:09,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17d18ef4424e487b359e HTTP/1.1" 200 None 2026-03-11 00:44:09,378 - lava_callback - DEBUG - Processing node 69ae17d18ef4424e487b359e: baseline-arm-baylibre 2026-03-11 00:44:09,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae17d18ef4424e487b359e 2026-03-11 00:44:09,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:09,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae17d18ef4424e487b359e HTTP/1.1" 200 None 2026-03-11 00:44:09,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:09,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-11 00:44:09,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:09,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae17d18ef4424e487b359e HTTP/1.1" 200 None 2026-03-11 00:44:09,434 - lava_callback - ERROR - Error processing callback for node 69ae17d18ef4424e487b359e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56624 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:44:40,927 - lava_callback - INFO - Received callback for node 69b0ba0c9d19175818252499 INFO: 172.21.0.1:42300 - "POST /node/69b0ba0c9d19175818252499 HTTP/1.0" 202 Accepted 2026-03-11 00:44:40,933 - lava_callback - INFO - Starting async job submit for node 69b0ba0c9d19175818252499 2026-03-11 00:44:40,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:40,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba0c9d19175818252499 HTTP/1.1" 200 None 2026-03-11 00:44:40,976 - lava_callback - DEBUG - Processing node 69b0ba0c9d19175818252499: ltp-containers 2026-03-11 00:44:41,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:44:41,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:44:41,532 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69b0ba0c9d19175818252499/log.txt.gz 2026-03-11 00:44:41,553 - lava_callback - INFO - Received callback for node 69b0b9ef9d19175818252438 INFO: 172.21.0.1:42308 - "POST /node/69b0b9ef9d19175818252438 HTTP/1.0" 202 Accepted 2026-03-11 00:44:41,576 - lava_callback - INFO - Starting async job submit for node 69b0b9ef9d19175818252438 2026-03-11 00:44:41,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:41,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ef9d19175818252438 HTTP/1.1" 200 None 2026-03-11 00:44:41,710 - lava_callback - DEBUG - Processing node 69b0b9ef9d19175818252438: baseline-arm-clabbe 2026-03-11 00:44:41,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:44:42,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:44:42,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:44:42,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9ef9d19175818252438/log.txt.gz 2026-03-11 00:44:42,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:44:42,284 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69b0ba0c9d19175818252499/lava-logs.yaml 2026-03-11 00:44:42,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:44:42,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:44:42,502 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:44:42,504 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9ef9d19175818252438/lava-logs.yaml 2026-03-11 00:44:42,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:44:42,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69b0ba0c9d19175818252499/lava_callback.json.gz 2026-03-11 00:44:42,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:42,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:44:42,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba0c9d19175818252499 HTTP/1.1" 200 None 2026-03-11 00:44:42,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:42,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:44:42,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:42,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba0c9d19175818252499 HTTP/1.1" 200 None 2026-03-11 00:44:42,618 - lava_callback - INFO - Completed processing callback for node 69b0ba0c9d19175818252499 2026-03-11 00:44:42,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:44:42,765 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9ef9d19175818252438/lava_callback.json.gz 2026-03-11 00:44:42,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:42,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9ef9d19175818252438 HTTP/1.1" 200 None 2026-03-11 00:44:42,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:42,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:44:42,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:42,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9ef9d19175818252438 HTTP/1.1" 200 None 2026-03-11 00:44:42,865 - lava_callback - INFO - Completed processing callback for node 69b0b9ef9d19175818252438 INFO: 172.21.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33292 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:44:56,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:44:56,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:45:01,172 - lava_callback - INFO - Received callback for node 69b0ba0d9d1917581825249c INFO: 172.21.0.1:55520 - "POST /node/69b0ba0d9d1917581825249c HTTP/1.0" 202 Accepted 2026-03-11 00:45:01,182 - lava_callback - INFO - Starting async job submit for node 69b0ba0d9d1917581825249c 2026-03-11 00:45:01,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:01,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba0d9d1917581825249c HTTP/1.1" 200 None 2026-03-11 00:45:01,255 - lava_callback - DEBUG - Processing node 69b0ba0d9d1917581825249c: ltp-controllers 2026-03-11 00:45:02,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:45:02,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:45:02,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69b0ba0d9d1917581825249c/log.txt.gz 2026-03-11 00:45:02,828 - lava_callback - INFO - Received callback for node 69b0ba1d9d191758182524dc INFO: 172.21.0.1:55536 - "POST /node/69b0ba1d9d191758182524dc HTTP/1.0" 202 Accepted 2026-03-11 00:45:02,850 - lava_callback - INFO - Starting async job submit for node 69b0ba1d9d191758182524dc 2026-03-11 00:45:02,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:02,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:45:02,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1d9d191758182524dc HTTP/1.1" 200 None 2026-03-11 00:45:02,985 - lava_callback - DEBUG - Processing node 69b0ba1d9d191758182524dc: ltp-syscalls 2026-03-11 00:45:03,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:45:03,539 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69b0ba0d9d1917581825249c/lava-logs.yaml 2026-03-11 00:45:03,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:45:04,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:45:04,147 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69b0ba0d9d1917581825249c/lava_callback.json.gz 2026-03-11 00:45:04,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:04,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba0d9d1917581825249c HTTP/1.1" 200 None 2026-03-11 00:45:04,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:04,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:45:04,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:04,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:45:04,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba0d9d1917581825249c HTTP/1.1" 200 None 2026-03-11 00:45:04,429 - lava_callback - INFO - Completed processing callback for node 69b0ba0d9d1917581825249c 2026-03-11 00:45:04,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:45:04,926 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b0ba1d9d191758182524dc/log.txt.gz 2026-03-11 00:45:05,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:45:06,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:45:06,177 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b0ba1d9d191758182524dc/lava-logs.yaml 2026-03-11 00:45:06,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:55538 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:45:06,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:45:06,653 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b0ba1d9d191758182524dc/lava_callback.json.gz 2026-03-11 00:45:06,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:06,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1d9d191758182524dc HTTP/1.1" 200 None 2026-03-11 00:45:06,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:06,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:45:06,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:06,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba1d9d191758182524dc HTTP/1.1" 200 None 2026-03-11 00:45:06,841 - lava_callback - INFO - Completed processing callback for node 69b0ba1d9d191758182524dc INFO: 172.21.0.1:41128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39030 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:45:26,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:45:26,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:46:24,629 - lava_callback - INFO - Received callback for node 69b0b9f09d1917581825243b INFO: 172.21.0.1:46010 - "POST /node/69b0b9f09d1917581825243b HTTP/1.0" 202 Accepted 2026-03-11 00:46:24,635 - lava_callback - INFO - Starting async job submit for node 69b0b9f09d1917581825243b 2026-03-11 00:46:24,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:24,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f09d1917581825243b HTTP/1.1" 200 None 2026-03-11 00:46:24,673 - lava_callback - DEBUG - Processing node 69b0b9f09d1917581825243b: baseline-arm-clabbe INFO: 172.21.0.1:46016 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:46:24,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:46:25,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:46:25,172 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9f09d1917581825243b/log.txt.gz 2026-03-11 00:46:25,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:46:25,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:46:25,653 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9f09d1917581825243b/lava-logs.yaml 2026-03-11 00:46:25,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:46:25,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:46:25,810 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b0b9f09d1917581825243b/lava_callback.json.gz 2026-03-11 00:46:25,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:25,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f09d1917581825243b HTTP/1.1" 200 None 2026-03-11 00:46:25,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:25,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:46:25,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:25,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9f09d1917581825243b HTTP/1.1" 200 None 2026-03-11 00:46:25,899 - lava_callback - INFO - Completed processing callback for node 69b0b9f09d1917581825243b INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:46:26,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:26,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55662 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:46:36,814 - lava_callback - INFO - Received callback for node 69b0ba109d191758182524a3 INFO: 172.21.0.1:55664 - "POST /node/69b0ba109d191758182524a3 HTTP/1.0" 202 Accepted 2026-03-11 00:46:36,820 - lava_callback - INFO - Starting async job submit for node 69b0ba109d191758182524a3 2026-03-11 00:46:36,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:36,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba109d191758182524a3 HTTP/1.1" 200 None 2026-03-11 00:46:36,863 - lava_callback - DEBUG - Processing node 69b0ba109d191758182524a3: ltp-cve 2026-03-11 00:46:37,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:46:37,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:46:37,429 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69b0ba109d191758182524a3/log.txt.gz 2026-03-11 00:46:37,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:46:37,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:46:37,873 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69b0ba109d191758182524a3/lava-logs.yaml 2026-03-11 00:46:37,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:46:38,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:46:38,080 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69b0ba109d191758182524a3/lava_callback.json.gz 2026-03-11 00:46:38,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:38,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba109d191758182524a3 HTTP/1.1" 200 None 2026-03-11 00:46:38,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:38,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:46:38,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:38,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba109d191758182524a3 HTTP/1.1" 200 None 2026-03-11 00:46:38,186 - lava_callback - INFO - Completed processing callback for node 69b0ba109d191758182524a3 INFO: 172.21.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60730 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:46:56,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:46:56,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:48:41,024 - lava_callback - INFO - Received callback for node 69b0ba139d191758182524a9 INFO: 172.21.0.1:57548 - "POST /node/69b0ba139d191758182524a9 HTTP/1.0" 202 Accepted 2026-03-11 00:48:41,031 - lava_callback - INFO - Starting async job submit for node 69b0ba139d191758182524a9 2026-03-11 00:48:41,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:41,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba139d191758182524a9 HTTP/1.1" 200 None 2026-03-11 00:48:41,074 - lava_callback - DEBUG - Processing node 69b0ba139d191758182524a9: ltp-fs_bind 2026-03-11 00:48:41,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:48:41,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:48:41,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b0ba139d191758182524a9/log.txt.gz 2026-03-11 00:48:41,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:48:42,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:48:42,048 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b0ba139d191758182524a9/lava-logs.yaml 2026-03-11 00:48:42,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:48:42,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:48:42,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b0ba139d191758182524a9/lava_callback.json.gz 2026-03-11 00:48:42,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:42,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba139d191758182524a9 HTTP/1.1" 200 None 2026-03-11 00:48:42,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:42,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:48:42,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:42,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba139d191758182524a9 HTTP/1.1" 200 None 2026-03-11 00:48:42,313 - lava_callback - INFO - Completed processing callback for node 69b0ba139d191758182524a9 INFO: 172.21.0.1:57556 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:48:49,660 - lava_callback - INFO - Received callback for node 69ae18eb8ef4424e487b374c INFO: 172.21.0.1:57570 - "POST /node/69ae18eb8ef4424e487b374c HTTP/1.0" 202 Accepted 2026-03-11 00:48:49,666 - lava_callback - INFO - Starting async job submit for node 69ae18eb8ef4424e487b374c 2026-03-11 00:48:49,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:49,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae18eb8ef4424e487b374c HTTP/1.1" 200 None 2026-03-11 00:48:49,683 - lava_callback - DEBUG - Processing node 69ae18eb8ef4424e487b374c: baseline-arm-baylibre 2026-03-11 00:48:49,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae18eb8ef4424e487b374c 2026-03-11 00:48:49,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:49,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae18eb8ef4424e487b374c HTTP/1.1" 200 None 2026-03-11 00:48:49,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:49,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-11 00:48:49,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:49,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae18eb8ef4424e487b374c HTTP/1.1" 200 None 2026-03-11 00:48:49,742 - lava_callback - ERROR - Error processing callback for node 69ae18eb8ef4424e487b374c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51184 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:48:56,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:48:56,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:49:01,172 - lava_callback - INFO - Received callback for node 69b0ba0b9d19175818252496 INFO: 172.21.0.1:56772 - "POST /node/69b0ba0b9d19175818252496 HTTP/1.0" 202 Accepted 2026-03-11 00:49:01,178 - lava_callback - INFO - Starting async job submit for node 69b0ba0b9d19175818252496 2026-03-11 00:49:01,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:01,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba0b9d19175818252496 HTTP/1.1" 200 None 2026-03-11 00:49:01,237 - lava_callback - DEBUG - Processing node 69b0ba0b9d19175818252496: ltp-cap_bounds 2026-03-11 00:49:01,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:01,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:01,875 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b0ba0b9d19175818252496/log.txt.gz 2026-03-11 00:49:02,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:02,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:02,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b0ba0b9d19175818252496/lava-logs.yaml 2026-03-11 00:49:02,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:02,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:02,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b0ba0b9d19175818252496/lava_callback.json.gz 2026-03-11 00:49:02,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:02,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba0b9d19175818252496 HTTP/1.1" 200 None 2026-03-11 00:49:02,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:02,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:49:02,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:02,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba0b9d19175818252496 HTTP/1.1" 200 None 2026-03-11 00:49:02,567 - lava_callback - INFO - Completed processing callback for node 69b0ba0b9d19175818252496 2026-03-11 00:49:02,745 - lava_callback - INFO - Received callback for node 69b0ba149d191758182524ac INFO: 172.21.0.1:56782 - "POST /node/69b0ba149d191758182524ac HTTP/1.0" 202 Accepted 2026-03-11 00:49:02,753 - lava_callback - INFO - Starting async job submit for node 69b0ba149d191758182524ac 2026-03-11 00:49:02,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:02,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba149d191758182524ac HTTP/1.1" 200 None 2026-03-11 00:49:02,810 - lava_callback - DEBUG - Processing node 69b0ba149d191758182524ac: ltp-fsx 2026-03-11 00:49:03,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:03,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:03,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69b0ba149d191758182524ac/log.txt.gz 2026-03-11 00:49:03,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:03,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:03,741 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69b0ba149d191758182524ac/lava-logs.yaml 2026-03-11 00:49:03,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:03,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:03,911 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69b0ba149d191758182524ac/lava_callback.json.gz 2026-03-11 00:49:03,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:03,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba149d191758182524ac HTTP/1.1" 200 None 2026-03-11 00:49:03,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:03,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:49:03,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:03,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba149d191758182524ac HTTP/1.1" 200 None 2026-03-11 00:49:04,001 - lava_callback - INFO - Completed processing callback for node 69b0ba149d191758182524ac INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35866 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-11 00:49:15,463 - lava_callback - INFO - Received callback for node 69b0b7899d1917581825201f INFO: 172.21.0.1:35882 - "POST /node/69b0b7899d1917581825201f HTTP/1.0" 202 Accepted 2026-03-11 00:49:15,471 - lava_callback - INFO - Starting async job submit for node 69b0b7899d1917581825201f 2026-03-11 00:49:15,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:15,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b7899d1917581825201f HTTP/1.1" 200 None 2026-03-11 00:49:15,543 - lava_callback - DEBUG - Processing node 69b0b7899d1917581825201f: ltp-smoke 2026-03-11 00:49:16,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:16,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:16,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0b7899d1917581825201f/log.txt.gz 2026-03-11 00:49:16,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35892 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:49:16,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:16,734 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0b7899d1917581825201f/lava-logs.yaml 2026-03-11 00:49:16,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:49:16,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:49:16,903 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0b7899d1917581825201f/lava_callback.json.gz 2026-03-11 00:49:17,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:17,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b7899d1917581825201f HTTP/1.1" 200 None 2026-03-11 00:49:17,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:17,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5059d19175818251fdc HTTP/1.1" 200 None 2026-03-11 00:49:17,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:17,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b7899d1917581825201f HTTP/1.1" 200 None 2026-03-11 00:49:17,202 - lava_callback - INFO - Completed processing callback for node 69b0b7899d1917581825201f INFO: 172.21.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49650 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:49:26,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:49:26,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35720 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:50:09,613 - lava_callback - INFO - Received callback for node 69ae193f8ef4424e487b37de INFO: 172.21.0.1:35726 - "POST /node/69ae193f8ef4424e487b37de HTTP/1.0" 202 Accepted 2026-03-11 00:50:09,618 - lava_callback - INFO - Starting async job submit for node 69ae193f8ef4424e487b37de 2026-03-11 00:50:09,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:09,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae193f8ef4424e487b37de HTTP/1.1" 200 None 2026-03-11 00:50:09,639 - lava_callback - DEBUG - Processing node 69ae193f8ef4424e487b37de: baseline-arm-baylibre 2026-03-11 00:50:09,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae193f8ef4424e487b37de 2026-03-11 00:50:09,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:09,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae193f8ef4424e487b37de HTTP/1.1" 200 None 2026-03-11 00:50:09,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:09,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628927da105ad09835991 HTTP/1.1" 200 None 2026-03-11 00:50:09,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:09,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae193f8ef4424e487b37de HTTP/1.1" 200 None 2026-03-11 00:50:09,694 - lava_callback - ERROR - Error processing callback for node 69ae193f8ef4424e487b37de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35900 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:50:55,745 - lava_callback - INFO - Received callback for node 69b0ba169d191758182524af INFO: 172.21.0.1:34660 - "POST /node/69b0ba169d191758182524af HTTP/1.0" 202 Accepted 2026-03-11 00:50:55,752 - lava_callback - INFO - Starting async job submit for node 69b0ba169d191758182524af 2026-03-11 00:50:55,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:55,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba169d191758182524af HTTP/1.1" 200 None 2026-03-11 00:50:55,792 - lava_callback - DEBUG - Processing node 69b0ba169d191758182524af: ltp-input 2026-03-11 00:50:56,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:50:56,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:50:56,335 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69b0ba169d191758182524af/log.txt.gz INFO: 172.21.0.1:34664 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:50:56,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:50:56,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:50:56,768 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69b0ba169d191758182524af/lava-logs.yaml 2026-03-11 00:50:56,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:50:56,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:50:56,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69b0ba169d191758182524af/lava_callback.json.gz 2026-03-11 00:50:56,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:56,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba169d191758182524af HTTP/1.1" 200 None 2026-03-11 00:50:56,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:56,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:50:56,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:50:57,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba169d191758182524af HTTP/1.1" 200 None 2026-03-11 00:50:57,032 - lava_callback - INFO - Completed processing callback for node 69b0ba169d191758182524af INFO: 172.21.0.1:39588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46334 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:51:26,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:51:26,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46346 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:51:40,126 - lava_callback - INFO - Received callback for node 69b0bc669d1917581825267f INFO: 172.21.0.1:36502 - "POST /node/69b0bc669d1917581825267f HTTP/1.0" 202 Accepted 2026-03-11 00:51:40,132 - lava_callback - INFO - Starting async job submit for node 69b0bc669d1917581825267f 2026-03-11 00:51:40,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:51:40,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc669d1917581825267f HTTP/1.1" 200 None 2026-03-11 00:51:40,165 - lava_callback - DEBUG - Processing node 69b0bc669d1917581825267f: baseline-arm-pengutronix 2026-03-11 00:51:40,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:51:40,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:51:40,451 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0bc669d1917581825267f/log.txt.gz 2026-03-11 00:51:40,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:51:40,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:51:40,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0bc669d1917581825267f/lava-logs.yaml 2026-03-11 00:51:40,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:51:40,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:51:40,907 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0bc669d1917581825267f/lava_callback.json.gz 2026-03-11 00:51:40,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:51:40,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bc669d1917581825267f HTTP/1.1" 200 None 2026-03-11 00:51:40,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:51:40,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:51:40,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:51:40,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bc669d1917581825267f HTTP/1.1" 200 None 2026-03-11 00:51:40,993 - lava_callback - INFO - Completed processing callback for node 69b0bc669d1917581825267f INFO: 172.21.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49950 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:51:56,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:51:56,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:52:41,161 - lava_callback - INFO - Received callback for node 69b0ba179d191758182524ca INFO: 172.21.0.1:57392 - "POST /node/69b0ba179d191758182524ca HTTP/1.0" 202 Accepted 2026-03-11 00:52:41,186 - lava_callback - INFO - Starting async job submit for node 69b0ba179d191758182524ca 2026-03-11 00:52:41,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:52:41,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba179d191758182524ca HTTP/1.1" 200 None 2026-03-11 00:52:41,278 - lava_callback - DEBUG - Processing node 69b0ba179d191758182524ca: ltp-ipc 2026-03-11 00:52:41,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:52:42,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:52:42,530 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69b0ba179d191758182524ca/log.txt.gz 2026-03-11 00:52:42,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:52:44,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:52:44,266 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69b0ba179d191758182524ca/lava-logs.yaml 2026-03-11 00:52:44,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:52:45,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:52:45,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69b0ba179d191758182524ca/lava_callback.json.gz 2026-03-11 00:52:45,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:52:45,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba179d191758182524ca HTTP/1.1" 200 None 2026-03-11 00:52:45,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:52:45,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:52:45,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:52:45,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba179d191758182524ca HTTP/1.1" 200 None 2026-03-11 00:52:45,333 - lava_callback - INFO - Completed processing callback for node 69b0ba179d191758182524ca INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46520 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:52:56,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:52:56,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:53:02,696 - lava_callback - INFO - Received callback for node 69b0ba199d191758182524d3 INFO: 172.21.0.1:50838 - "POST /node/69b0ba199d191758182524d3 HTTP/1.0" 202 Accepted 2026-03-11 00:53:02,706 - lava_callback - INFO - Starting async job submit for node 69b0ba199d191758182524d3 2026-03-11 00:53:02,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:02,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba199d191758182524d3 HTTP/1.1" 200 None 2026-03-11 00:53:02,788 - lava_callback - DEBUG - Processing node 69b0ba199d191758182524d3: ltp-pty 2026-03-11 00:53:03,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:53:04,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:53:04,273 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69b0ba199d191758182524d3/log.txt.gz 2026-03-11 00:53:04,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:53:05,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:53:05,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69b0ba199d191758182524d3/lava-logs.yaml 2026-03-11 00:53:05,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50854 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:53:07,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:53:07,053 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69b0ba199d191758182524d3/lava_callback.json.gz 2026-03-11 00:53:07,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:07,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba199d191758182524d3 HTTP/1.1" 200 None 2026-03-11 00:53:07,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:07,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:53:07,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:07,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba199d191758182524d3 HTTP/1.1" 200 None 2026-03-11 00:53:07,237 - lava_callback - INFO - Completed processing callback for node 69b0ba199d191758182524d3 INFO: 172.21.0.1:58770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57388 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:53:26,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:26,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43344 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:53:41,231 - lava_callback - INFO - Received callback for node 69b0ba1a9d191758182524d6 INFO: 172.21.0.1:35232 - "POST /node/69b0ba1a9d191758182524d6 HTTP/1.0" 202 Accepted 2026-03-11 00:53:41,242 - lava_callback - INFO - Starting async job submit for node 69b0ba1a9d191758182524d6 2026-03-11 00:53:41,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:41,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1a9d191758182524d6 HTTP/1.1" 200 None 2026-03-11 00:53:41,320 - lava_callback - DEBUG - Processing node 69b0ba1a9d191758182524d6: ltp-sched 2026-03-11 00:53:42,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:53:43,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:53:43,130 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69b0ba1a9d191758182524d6/log.txt.gz 2026-03-11 00:53:43,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:53:45,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:53:45,629 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69b0ba1a9d191758182524d6/lava-logs.yaml 2026-03-11 00:53:45,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:35234 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:53:47,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:53:47,204 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69b0ba1a9d191758182524d6/lava_callback.json.gz 2026-03-11 00:53:47,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:47,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1a9d191758182524d6 HTTP/1.1" 200 None 2026-03-11 00:53:47,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:47,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:53:47,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:47,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba1a9d191758182524d6 HTTP/1.1" 200 None 2026-03-11 00:53:47,382 - lava_callback - INFO - Completed processing callback for node 69b0ba1a9d191758182524d6 INFO: 172.21.0.1:48638 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:53:56,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:53:56,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51968 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:54:12,938 - lava_callback - INFO - Received callback for node 69b0bd1d9d191758182526d3 INFO: 172.21.0.1:36410 - "POST /node/69b0bd1d9d191758182526d3 HTTP/1.0" 202 Accepted 2026-03-11 00:54:12,949 - lava_callback - INFO - Starting async job submit for node 69b0bd1d9d191758182526d3 2026-03-11 00:54:12,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:54:12,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bd1d9d191758182526d3 HTTP/1.1" 200 None 2026-03-11 00:54:12,979 - lava_callback - DEBUG - Processing node 69b0bd1d9d191758182526d3: baseline-arm 2026-03-11 00:54:12,993 - lava_callback - WARNING - Failed to upload log 2026-03-11 00:54:12,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:54:13,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:54:13,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0bd1d9d191758182526d3/lava-logs.yaml 2026-03-11 00:54:13,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:54:14,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:54:14,941 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0bd1d9d191758182526d3/lava_callback.json.gz 2026-03-11 00:54:14,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:54:14,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bd1d9d191758182526d3 HTTP/1.1" 200 None 2026-03-11 00:54:14,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:54:14,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:54:14,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:54:15,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bd1d9d191758182526d3 HTTP/1.1" 200 None 2026-03-11 00:54:15,062 - lava_callback - INFO - Completed processing callback for node 69b0bd1d9d191758182526d3 INFO: 172.21.0.1:36418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:54:26,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:54:27,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49436 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:55:17,329 - lava_callback - INFO - Received callback for node 69b0ba1c9d191758182524d9 INFO: 172.21.0.1:49444 - "POST /node/69b0ba1c9d191758182524d9 HTTP/1.0" 202 Accepted 2026-03-11 00:55:17,341 - lava_callback - INFO - Starting async job submit for node 69b0ba1c9d191758182524d9 2026-03-11 00:55:17,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:17,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1c9d191758182524d9 HTTP/1.1" 200 None 2026-03-11 00:55:17,423 - lava_callback - DEBUG - Processing node 69b0ba1c9d191758182524d9: ltp-smoke 2026-03-11 00:55:18,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:55:18,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:55:18,772 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0ba1c9d191758182524d9/log.txt.gz 2026-03-11 00:55:19,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:55:20,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:55:20,823 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0ba1c9d191758182524d9/lava-logs.yaml 2026-03-11 00:55:20,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:55:21,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:55:21,993 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0ba1c9d191758182524d9/lava_callback.json.gz 2026-03-11 00:55:22,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:22,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1c9d191758182524d9 HTTP/1.1" 200 None 2026-03-11 00:55:22,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:22,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:55:22,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:22,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba1c9d191758182524d9 HTTP/1.1" 200 None 2026-03-11 00:55:22,141 - lava_callback - INFO - Completed processing callback for node 69b0ba1c9d191758182524d9 INFO: 172.21.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50474 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:55:27,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:27,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52596 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:55:49,962 - lava_callback - INFO - Received callback for node 69ae1a888ef4424e487b39d5 INFO: 172.21.0.1:55064 - "POST /node/69ae1a888ef4424e487b39d5 HTTP/1.0" 202 Accepted 2026-03-11 00:55:49,971 - lava_callback - INFO - Starting async job submit for node 69ae1a888ef4424e487b39d5 2026-03-11 00:55:49,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:50,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a888ef4424e487b39d5 HTTP/1.1" 200 None 2026-03-11 00:55:50,015 - lava_callback - DEBUG - Processing node 69ae1a888ef4424e487b39d5: baseline-arm-baylibre 2026-03-11 00:55:50,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1a888ef4424e487b39d5 2026-03-11 00:55:50,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:50,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a888ef4424e487b39d5 HTTP/1.1" 200 None 2026-03-11 00:55:50,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:50,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 00:55:50,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:55:50,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a888ef4424e487b39d5 HTTP/1.1" 200 None 2026-03-11 00:55:50,125 - lava_callback - ERROR - Error processing callback for node 69ae1a888ef4424e487b39d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:56:29,916 - lava_callback - INFO - Received callback for node 69ae1ab38ef4424e487b39ff INFO: 172.21.0.1:39664 - "POST /node/69ae1ab38ef4424e487b39ff HTTP/1.0" 202 Accepted 2026-03-11 00:56:29,926 - lava_callback - INFO - Starting async job submit for node 69ae1ab38ef4424e487b39ff 2026-03-11 00:56:29,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:29,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ab38ef4424e487b39ff HTTP/1.1" 200 None 2026-03-11 00:56:29,985 - lava_callback - DEBUG - Processing node 69ae1ab38ef4424e487b39ff: baseline-arm-baylibre 2026-03-11 00:56:29,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1ab38ef4424e487b39ff 2026-03-11 00:56:29,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:30,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ab38ef4424e487b39ff HTTP/1.1" 200 None 2026-03-11 00:56:30,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:30,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 00:56:30,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:30,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ab38ef4424e487b39ff HTTP/1.1" 200 None 2026-03-11 00:56:30,171 - lava_callback - ERROR - Error processing callback for node 69ae1ab38ef4424e487b39ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39672 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:56:41,351 - lava_callback - INFO - Received callback for node 69b0ba1f9d191758182524df INFO: 172.21.0.1:48290 - "POST /node/69b0ba1f9d191758182524df HTTP/1.0" 202 Accepted 2026-03-11 00:56:41,363 - lava_callback - INFO - Starting async job submit for node 69b0ba1f9d191758182524df 2026-03-11 00:56:41,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:41,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1f9d191758182524df HTTP/1.1" 200 None 2026-03-11 00:56:41,457 - lava_callback - DEBUG - Processing node 69b0ba1f9d191758182524df: ltp-syscalls-ipc 2026-03-11 00:56:42,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:56:43,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:56:43,178 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b0ba1f9d191758182524df/log.txt.gz 2026-03-11 00:56:43,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:56:45,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:56:45,296 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b0ba1f9d191758182524df/lava-logs.yaml 2026-03-11 00:56:45,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:56:46,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:56:46,487 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b0ba1f9d191758182524df/lava_callback.json.gz INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:56:46,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:46,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba1f9d191758182524df HTTP/1.1" 200 None 2026-03-11 00:56:46,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:46,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:56:46,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:46,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba1f9d191758182524df HTTP/1.1" 200 None 2026-03-11 00:56:46,634 - lava_callback - INFO - Completed processing callback for node 69b0ba1f9d191758182524df INFO: 172.21.0.1:34316 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:56:57,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:56:57,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:57:01,327 - lava_callback - INFO - Received callback for node 69b0ba219d191758182524e2 INFO: 172.21.0.1:48718 - "POST /node/69b0ba219d191758182524e2 HTTP/1.0" 202 Accepted 2026-03-11 00:57:01,337 - lava_callback - INFO - Starting async job submit for node 69b0ba219d191758182524e2 2026-03-11 00:57:01,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:01,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba219d191758182524e2 HTTP/1.1" 200 None 2026-03-11 00:57:01,416 - lava_callback - DEBUG - Processing node 69b0ba219d191758182524e2: ltp-timers 2026-03-11 00:57:02,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:57:02,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:57:02,686 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69b0ba219d191758182524e2/log.txt.gz 2026-03-11 00:57:03,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:57:04,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:57:04,368 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69b0ba219d191758182524e2/lava-logs.yaml 2026-03-11 00:57:04,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:57:05,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:57:05,243 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69b0ba219d191758182524e2/lava_callback.json.gz 2026-03-11 00:57:05,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:05,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba219d191758182524e2 HTTP/1.1" 200 None 2026-03-11 00:57:05,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:05,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:57:05,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:05,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba219d191758182524e2 HTTP/1.1" 200 None 2026-03-11 00:57:05,411 - lava_callback - INFO - Completed processing callback for node 69b0ba219d191758182524e2 INFO: 172.21.0.1:48734 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:57:09,989 - lava_callback - INFO - Received callback for node 69ae1ade8ef4424e487b3a32 INFO: 172.21.0.1:38734 - "POST /node/69ae1ade8ef4424e487b3a32 HTTP/1.0" 202 Accepted 2026-03-11 00:57:09,998 - lava_callback - INFO - Starting async job submit for node 69ae1ade8ef4424e487b3a32 2026-03-11 00:57:09,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:10,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ade8ef4424e487b3a32 HTTP/1.1" 200 None 2026-03-11 00:57:10,034 - lava_callback - DEBUG - Processing node 69ae1ade8ef4424e487b3a32: baseline-arm-baylibre 2026-03-11 00:57:10,034 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1ade8ef4424e487b3a32 2026-03-11 00:57:10,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:10,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1ade8ef4424e487b3a32 HTTP/1.1" 200 None 2026-03-11 00:57:10,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:10,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 00:57:10,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:10,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1ade8ef4424e487b3a32 HTTP/1.1" 200 None 2026-03-11 00:57:10,129 - lava_callback - ERROR - Error processing callback for node 69ae1ade8ef4424e487b3a32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36560 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:57:27,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:27,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35650 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:57:49,979 - lava_callback - INFO - Received callback for node 69ae1b088ef4424e487b3a7d INFO: 172.21.0.1:60176 - "POST /node/69ae1b088ef4424e487b3a7d HTTP/1.0" 202 Accepted 2026-03-11 00:57:49,988 - lava_callback - INFO - Starting async job submit for node 69ae1b088ef4424e487b3a7d 2026-03-11 00:57:49,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:50,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b088ef4424e487b3a7d HTTP/1.1" 200 None 2026-03-11 00:57:50,019 - lava_callback - DEBUG - Processing node 69ae1b088ef4424e487b3a7d: baseline-arm-baylibre 2026-03-11 00:57:50,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1b088ef4424e487b3a7d 2026-03-11 00:57:50,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:50,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b088ef4424e487b3a7d HTTP/1.1" 200 None 2026-03-11 00:57:50,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:50,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 00:57:50,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:57:50,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1b088ef4424e487b3a7d HTTP/1.1" 200 None 2026-03-11 00:57:50,118 - lava_callback - ERROR - Error processing callback for node 69ae1b088ef4424e487b3a7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:58:08,555 - lava_callback - INFO - Received callback for node 69b0bdfc9d19175818252758 INFO: 172.21.0.1:43168 - "POST /node/69b0bdfc9d19175818252758 HTTP/1.0" 202 Accepted 2026-03-11 00:58:08,566 - lava_callback - INFO - Starting async job submit for node 69b0bdfc9d19175818252758 2026-03-11 00:58:08,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:08,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdfc9d19175818252758 HTTP/1.1" 200 None 2026-03-11 00:58:08,677 - lava_callback - DEBUG - Processing node 69b0bdfc9d19175818252758: baseline-arm64 2026-03-11 00:58:08,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:09,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:09,472 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfc9d19175818252758/log.txt.gz 2026-03-11 00:58:09,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:10,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:10,403 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfc9d19175818252758/lava-logs.yaml 2026-03-11 00:58:10,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:10,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:10,864 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfc9d19175818252758/lava_callback.json.gz 2026-03-11 00:58:10,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:10,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdfc9d19175818252758 HTTP/1.1" 200 None 2026-03-11 00:58:10,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:10,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:58:10,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:11,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bdfc9d19175818252758 HTTP/1.1" 200 None 2026-03-11 00:58:11,042 - lava_callback - INFO - Completed processing callback for node 69b0bdfc9d19175818252758 INFO: 172.21.0.1:50470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50472 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:58:21,391 - lava_callback - INFO - Received callback for node 69b0ba239d191758182524e5 INFO: 172.21.0.1:36138 - "POST /node/69b0ba239d191758182524e5 HTTP/1.0" 202 Accepted 2026-03-11 00:58:21,398 - lava_callback - INFO - Starting async job submit for node 69b0ba239d191758182524e5 2026-03-11 00:58:21,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:21,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba239d191758182524e5 HTTP/1.1" 200 None 2026-03-11 00:58:21,439 - lava_callback - DEBUG - Processing node 69b0ba239d191758182524e5: ltp-watchqueue 2026-03-11 00:58:21,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:22,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:22,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b0ba239d191758182524e5/log.txt.gz 2026-03-11 00:58:22,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:22,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:22,444 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b0ba239d191758182524e5/lava-logs.yaml 2026-03-11 00:58:22,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:22,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:22,597 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b0ba239d191758182524e5/lava_callback.json.gz 2026-03-11 00:58:22,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:22,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ba239d191758182524e5 HTTP/1.1" 200 None 2026-03-11 00:58:22,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:22,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:58:22,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:22,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ba239d191758182524e5 HTTP/1.1" 200 None 2026-03-11 00:58:22,679 - lava_callback - INFO - Completed processing callback for node 69b0ba239d191758182524e5 INFO: 172.21.0.1:36140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36152 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:58:27,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:27,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:58:28,791 - lava_callback - INFO - Received callback for node 69b0be1e9d191758182527c4 INFO: 172.21.0.1:36168 - "POST /node/69b0be1e9d191758182527c4 HTTP/1.0" 202 Accepted 2026-03-11 00:58:28,798 - lava_callback - INFO - Starting async job submit for node 69b0be1e9d191758182527c4 2026-03-11 00:58:28,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:28,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1e9d191758182527c4 HTTP/1.1" 200 None 2026-03-11 00:58:28,831 - lava_callback - DEBUG - Processing node 69b0be1e9d191758182527c4: baseline-nfs-arm64 2026-03-11 00:58:28,833 - lava_callback - WARNING - Failed to upload log 2026-03-11 00:58:28,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:29,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:29,017 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1e9d191758182527c4/lava-logs.yaml 2026-03-11 00:58:29,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:29,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:29,137 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1e9d191758182527c4/lava_callback.json.gz 2026-03-11 00:58:29,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:29,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1e9d191758182527c4 HTTP/1.1" 200 None 2026-03-11 00:58:29,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:29,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:58:29,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:29,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be1e9d191758182527c4 HTTP/1.1" 200 None 2026-03-11 00:58:29,211 - lava_callback - INFO - Completed processing callback for node 69b0be1e9d191758182527c4 2026-03-11 00:58:30,105 - lava_callback - INFO - Received callback for node 69b0be0b9d1917581825277c INFO: 172.21.0.1:55652 - "POST /node/69b0be0b9d1917581825277c HTTP/1.0" 202 Accepted 2026-03-11 00:58:30,112 - lava_callback - INFO - Starting async job submit for node 69b0be0b9d1917581825277c 2026-03-11 00:58:30,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:30,131 - lava_callback - INFO - Received callback for node 69ae1b328ef4424e487b3aab INFO: 172.21.0.1:55654 - "POST /node/69ae1b328ef4424e487b3aab HTTP/1.0" 202 Accepted 2026-03-11 00:58:30,137 - lava_callback - INFO - Starting async job submit for node 69ae1b328ef4424e487b3aab 2026-03-11 00:58:30,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:30,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0b9d1917581825277c HTTP/1.1" 200 None 2026-03-11 00:58:30,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b328ef4424e487b3aab HTTP/1.1" 200 None 2026-03-11 00:58:30,154 - lava_callback - DEBUG - Processing node 69b0be0b9d1917581825277c: baseline-arm64 2026-03-11 00:58:30,160 - lava_callback - DEBUG - Processing node 69ae1b328ef4424e487b3aab: baseline-arm-baylibre 2026-03-11 00:58:30,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1b328ef4424e487b3aab 2026-03-11 00:58:30,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:30,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b328ef4424e487b3aab HTTP/1.1" 200 None 2026-03-11 00:58:30,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:30,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 00:58:30,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:30,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:30,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1b328ef4424e487b3aab HTTP/1.1" 200 None 2026-03-11 00:58:30,421 - lava_callback - ERROR - Error processing callback for node 69ae1b328ef4424e487b3aab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:58:30,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:30,588 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0b9d1917581825277c/log.txt.gz 2026-03-11 00:58:30,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:30,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:30,894 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0b9d1917581825277c/lava-logs.yaml 2026-03-11 00:58:30,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:31,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:31,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0b9d1917581825277c/lava_callback.json.gz 2026-03-11 00:58:31,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:31,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0b9d1917581825277c HTTP/1.1" 200 None 2026-03-11 00:58:31,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:31,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:58:31,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:31,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be0b9d1917581825277c HTTP/1.1" 200 None 2026-03-11 00:58:31,130 - lava_callback - INFO - Completed processing callback for node 69b0be0b9d1917581825277c 2026-03-11 00:58:33,141 - lava_callback - INFO - Received callback for node 69b0be239d191758182527d3 INFO: 172.21.0.1:55668 - "POST /node/69b0be239d191758182527d3 HTTP/1.0" 202 Accepted 2026-03-11 00:58:33,148 - lava_callback - INFO - Starting async job submit for node 69b0be239d191758182527d3 2026-03-11 00:58:33,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:33,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be239d191758182527d3 HTTP/1.1" 200 None 2026-03-11 00:58:33,168 - lava_callback - DEBUG - Processing node 69b0be239d191758182527d3: baseline-arm 2026-03-11 00:58:33,174 - lava_callback - WARNING - Failed to upload log 2026-03-11 00:58:33,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:33,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:33,299 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0be239d191758182527d3/lava-logs.yaml 2026-03-11 00:58:33,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:33,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:33,420 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0be239d191758182527d3/lava_callback.json.gz 2026-03-11 00:58:33,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:33,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be239d191758182527d3 HTTP/1.1" 200 None 2026-03-11 00:58:33,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:33,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:58:33,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:33,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be239d191758182527d3 HTTP/1.1" 200 None 2026-03-11 00:58:33,553 - lava_callback - INFO - Completed processing callback for node 69b0be239d191758182527d3 INFO: 172.21.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:58:49,143 - lava_callback - INFO - Received callback for node 69b0bdfe9d1917581825275e INFO: 172.21.0.1:55496 - "POST /node/69b0bdfe9d1917581825275e HTTP/1.0" 202 Accepted 2026-03-11 00:58:49,154 - lava_callback - INFO - Starting async job submit for node 69b0bdfe9d1917581825275e 2026-03-11 00:58:49,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:49,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdfe9d1917581825275e HTTP/1.1" 200 None 2026-03-11 00:58:49,241 - lava_callback - DEBUG - Processing node 69b0bdfe9d1917581825275e: baseline-arm64 2026-03-11 00:58:49,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:49,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:49,690 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfe9d1917581825275e/log.txt.gz 2026-03-11 00:58:49,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:50,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:50,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfe9d1917581825275e/lava-logs.yaml 2026-03-11 00:58:50,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:50,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:50,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfe9d1917581825275e/lava_callback.json.gz 2026-03-11 00:58:50,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:50,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdfe9d1917581825275e HTTP/1.1" 200 None 2026-03-11 00:58:50,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:50,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:58:50,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:50,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bdfe9d1917581825275e HTTP/1.1" 200 None 2026-03-11 00:58:50,357 - lava_callback - INFO - Completed processing callback for node 69b0bdfe9d1917581825275e 2026-03-11 00:58:56,174 - lava_callback - INFO - Received callback for node 69b0be029d19175818252767 INFO: 172.21.0.1:34966 - "POST /node/69b0be029d19175818252767 HTTP/1.0" 202 Accepted 2026-03-11 00:58:56,186 - lava_callback - INFO - Starting async job submit for node 69b0be029d19175818252767 2026-03-11 00:58:56,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:56,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be029d19175818252767 HTTP/1.1" 200 None 2026-03-11 00:58:56,273 - lava_callback - DEBUG - Processing node 69b0be029d19175818252767: baseline-arm64 2026-03-11 00:58:56,498 - lava_callback - WARNING - Failed to upload log INFO: 172.21.0.1:34972 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:58:56,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:56,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:56,764 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be029d19175818252767/lava-logs.yaml 2026-03-11 00:58:56,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:56,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:56,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be029d19175818252767/lava_callback.json.gz 2026-03-11 00:58:56,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:56,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be029d19175818252767 HTTP/1.1" 200 None 2026-03-11 00:58:56,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:56,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:58:56,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:56,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be029d19175818252767 HTTP/1.1" 200 None 2026-03-11 00:58:57,054 - lava_callback - INFO - Completed processing callback for node 69b0be029d19175818252767 2026-03-11 00:58:57,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:57,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 00:58:59,229 - lava_callback - INFO - Received callback for node 69b0be019d19175818252764 INFO: 172.21.0.1:34984 - "POST /node/69b0be019d19175818252764 HTTP/1.0" 202 Accepted 2026-03-11 00:58:59,236 - lava_callback - INFO - Starting async job submit for node 69b0be019d19175818252764 2026-03-11 00:58:59,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:59,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be019d19175818252764 HTTP/1.1" 200 None 2026-03-11 00:58:59,284 - lava_callback - DEBUG - Processing node 69b0be019d19175818252764: baseline-arm64 2026-03-11 00:58:59,436 - lava_callback - WARNING - Failed to upload log 2026-03-11 00:58:59,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:59,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:59,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be019d19175818252764/lava-logs.yaml 2026-03-11 00:58:59,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:58:59,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:58:59,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be019d19175818252764/lava_callback.json.gz 2026-03-11 00:58:59,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:59,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be019d19175818252764 HTTP/1.1" 200 None 2026-03-11 00:58:59,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:59,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:58:59,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:58:59,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be019d19175818252764 HTTP/1.1" 200 None 2026-03-11 00:58:59,992 - lava_callback - INFO - Completed processing callback for node 69b0be019d19175818252764 INFO: 172.21.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32872 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:59:19,927 - lava_callback - INFO - Received callback for node 69b0be259d191758182527da INFO: 172.21.0.1:51846 - "POST /node/69b0be259d191758182527da HTTP/1.0" 202 Accepted 2026-03-11 00:59:19,934 - lava_callback - INFO - Starting async job submit for node 69b0be259d191758182527da 2026-03-11 00:59:19,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:19,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be259d191758182527da HTTP/1.1" 200 None 2026-03-11 00:59:19,973 - lava_callback - DEBUG - Processing node 69b0be259d191758182527da: baseline-nfs-arm64 2026-03-11 00:59:20,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:20,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:20,530 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be259d191758182527da/log.txt.gz 2026-03-11 00:59:20,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:20,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:20,943 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be259d191758182527da/lava-logs.yaml 2026-03-11 00:59:20,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:21,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:21,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be259d191758182527da/lava_callback.json.gz 2026-03-11 00:59:21,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be259d191758182527da HTTP/1.1" 200 None 2026-03-11 00:59:21,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:21,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be259d191758182527da HTTP/1.1" 200 None 2026-03-11 00:59:21,205 - lava_callback - INFO - Completed processing callback for node 69b0be259d191758182527da 2026-03-11 00:59:21,586 - lava_callback - INFO - Received callback for node 69b0bdfd9d1917581825275b INFO: 172.21.0.1:51854 - "POST /node/69b0bdfd9d1917581825275b HTTP/1.0" 202 Accepted 2026-03-11 00:59:21,592 - lava_callback - INFO - Starting async job submit for node 69b0bdfd9d1917581825275b 2026-03-11 00:59:21,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdfd9d1917581825275b HTTP/1.1" 200 None 2026-03-11 00:59:21,647 - lava_callback - DEBUG - Processing node 69b0bdfd9d1917581825275b: baseline-arm64 2026-03-11 00:59:21,679 - lava_callback - INFO - Received callback for node 69ae1b5d8ef4424e487b3afc INFO: 172.21.0.1:51870 - "POST /node/69ae1b5d8ef4424e487b3afc HTTP/1.0" 202 Accepted 2026-03-11 00:59:21,705 - lava_callback - INFO - Starting async job submit for node 69ae1b5d8ef4424e487b3afc 2026-03-11 00:59:21,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b5d8ef4424e487b3afc HTTP/1.1" 200 None 2026-03-11 00:59:21,816 - lava_callback - DEBUG - Processing node 69ae1b5d8ef4424e487b3afc: baseline-arm-clabbe 2026-03-11 00:59:21,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1b5d8ef4424e487b3afc 2026-03-11 00:59:21,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:21,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b5d8ef4424e487b3afc HTTP/1.1" 200 None 2026-03-11 00:59:21,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 00:59:21,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:21,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1b5d8ef4424e487b3afc HTTP/1.1" 200 None 2026-03-11 00:59:21,947 - lava_callback - ERROR - Error processing callback for node 69ae1b5d8ef4424e487b3afc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 00:59:22,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:22,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfd9d1917581825275b/log.txt.gz 2026-03-11 00:59:22,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:22,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:22,377 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfd9d1917581825275b/lava-logs.yaml 2026-03-11 00:59:22,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:22,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:22,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdfd9d1917581825275b/lava_callback.json.gz 2026-03-11 00:59:22,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:22,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdfd9d1917581825275b HTTP/1.1" 200 None 2026-03-11 00:59:22,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:22,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:22,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:22,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bdfd9d1917581825275b HTTP/1.1" 200 None 2026-03-11 00:59:22,641 - lava_callback - INFO - Completed processing callback for node 69b0bdfd9d1917581825275b 2026-03-11 00:59:24,284 - lava_callback - INFO - Received callback for node 69b0be119d1917581825278a INFO: 172.21.0.1:51878 - "POST /node/69b0be119d1917581825278a HTTP/1.0" 202 Accepted 2026-03-11 00:59:24,290 - lava_callback - INFO - Starting async job submit for node 69b0be119d1917581825278a 2026-03-11 00:59:24,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:24,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be119d1917581825278a HTTP/1.1" 200 None 2026-03-11 00:59:24,327 - lava_callback - DEBUG - Processing node 69b0be119d1917581825278a: baseline-arm64 2026-03-11 00:59:24,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:24,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:24,695 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be119d1917581825278a/log.txt.gz 2026-03-11 00:59:24,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:25,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:25,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be119d1917581825278a/lava-logs.yaml 2026-03-11 00:59:25,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:51886 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:59:25,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:25,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be119d1917581825278a/lava_callback.json.gz 2026-03-11 00:59:25,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:25,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be119d1917581825278a HTTP/1.1" 200 None 2026-03-11 00:59:25,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:25,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:25,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:25,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be119d1917581825278a HTTP/1.1" 200 None 2026-03-11 00:59:25,256 - lava_callback - INFO - Completed processing callback for node 69b0be119d1917581825278a INFO: 172.21.0.1:51894 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:59:27,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:27,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-11 00:59:33,192 - lava_callback - INFO - Received callback for node 69b0be059d19175818252770 INFO: 172.21.0.1:57692 - "POST /node/69b0be059d19175818252770 HTTP/1.0" 202 Accepted 2026-03-11 00:59:33,199 - lava_callback - INFO - Starting async job submit for node 69b0be059d19175818252770 2026-03-11 00:59:33,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:33,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be059d19175818252770 HTTP/1.1" 200 None 2026-03-11 00:59:33,235 - lava_callback - DEBUG - Processing node 69b0be059d19175818252770: baseline-arm64 2026-03-11 00:59:33,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:33,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:33,597 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be059d19175818252770/log.txt.gz 2026-03-11 00:59:33,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:33,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:33,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be059d19175818252770/lava-logs.yaml 2026-03-11 00:59:33,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:34,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:34,065 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be059d19175818252770/lava_callback.json.gz 2026-03-11 00:59:34,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:34,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be059d19175818252770 HTTP/1.1" 200 None 2026-03-11 00:59:34,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:34,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:34,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:34,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be059d19175818252770 HTTP/1.1" 200 None 2026-03-11 00:59:34,151 - lava_callback - INFO - Completed processing callback for node 69b0be059d19175818252770 INFO: 172.21.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41048 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:59:47,533 - lava_callback - INFO - Received callback for node 69b0b9f99d19175818252455 INFO: 172.21.0.1:41052 - "POST /node/69b0b9f99d19175818252455 HTTP/1.0" 202 Accepted 2026-03-11 00:59:47,581 - lava_callback - INFO - Starting async job submit for node 69b0b9f99d19175818252455 2026-03-11 00:59:47,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:47,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f99d19175818252455 HTTP/1.1" 200 None 2026-03-11 00:59:47,622 - lava_callback - DEBUG - Processing node 69b0b9f99d19175818252455: baseline-arm-pengutronix 2026-03-11 00:59:47,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:47,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:47,855 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f99d19175818252455/log.txt.gz 2026-03-11 00:59:47,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:48,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:48,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f99d19175818252455/lava-logs.yaml 2026-03-11 00:59:48,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:48,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:48,248 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0b9f99d19175818252455/lava_callback.json.gz DEBUG: Setup failed for 69b0b9f99d19175818252455. Transitting job node result: incomplete -> fail 2026-03-11 00:59:48,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:48,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9f99d19175818252455 HTTP/1.1" 200 None 2026-03-11 00:59:48,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:48,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 00:59:48,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:48,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9f99d19175818252455 HTTP/1.1" 200 None 2026-03-11 00:59:48,433 - lava_callback - INFO - Completed processing callback for node 69b0b9f99d19175818252455 2026-03-11 00:59:49,209 - lava_callback - INFO - Received callback for node 69b0be4b9d1917581825285b INFO: 172.21.0.1:41062 - "POST /node/69b0be4b9d1917581825285b HTTP/1.0" 202 Accepted 2026-03-11 00:59:49,219 - lava_callback - INFO - Starting async job submit for node 69b0be4b9d1917581825285b 2026-03-11 00:59:49,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:49,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4b9d1917581825285b HTTP/1.1" 200 None 2026-03-11 00:59:49,261 - lava_callback - DEBUG - Processing node 69b0be4b9d1917581825285b: baseline-arm64-pengutronix 2026-03-11 00:59:49,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:49,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:49,633 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4b9d1917581825285b/log.txt.gz 2026-03-11 00:59:49,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:49,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:49,924 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4b9d1917581825285b/lava-logs.yaml 2026-03-11 00:59:49,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:50,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:50,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4b9d1917581825285b/lava_callback.json.gz 2026-03-11 00:59:50,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:50,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4b9d1917581825285b HTTP/1.1" 200 None 2026-03-11 00:59:50,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:50,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:50,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:50,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be4b9d1917581825285b HTTP/1.1" 200 None 2026-03-11 00:59:50,235 - lava_callback - INFO - Completed processing callback for node 69b0be4b9d1917581825285b 2026-03-11 00:59:53,240 - lava_callback - INFO - Received callback for node 69b0be089d19175818252776 INFO: 172.21.0.1:35568 - "POST /node/69b0be089d19175818252776 HTTP/1.0" 202 Accepted 2026-03-11 00:59:53,246 - lava_callback - INFO - Starting async job submit for node 69b0be089d19175818252776 2026-03-11 00:59:53,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:53,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be089d19175818252776 HTTP/1.1" 200 None 2026-03-11 00:59:53,284 - lava_callback - DEBUG - Processing node 69b0be089d19175818252776: baseline-arm64 2026-03-11 00:59:53,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:53,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:53,784 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be089d19175818252776/log.txt.gz 2026-03-11 00:59:54,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:54,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:54,204 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be089d19175818252776/lava-logs.yaml 2026-03-11 00:59:54,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:54,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:54,367 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be089d19175818252776/lava_callback.json.gz 2026-03-11 00:59:54,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:54,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be089d19175818252776 HTTP/1.1" 200 None 2026-03-11 00:59:54,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:54,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:54,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:54,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be089d19175818252776 HTTP/1.1" 200 None 2026-03-11 00:59:54,446 - lava_callback - INFO - Completed processing callback for node 69b0be089d19175818252776 2026-03-11 00:59:54,718 - lava_callback - INFO - Received callback for node 69b0be079d19175818252773 INFO: 172.21.0.1:35580 - "POST /node/69b0be079d19175818252773 HTTP/1.0" 202 Accepted 2026-03-11 00:59:54,727 - lava_callback - INFO - Starting async job submit for node 69b0be079d19175818252773 2026-03-11 00:59:54,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:54,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be079d19175818252773 HTTP/1.1" 200 None 2026-03-11 00:59:54,776 - lava_callback - DEBUG - Processing node 69b0be079d19175818252773: baseline-arm64 2026-03-11 00:59:55,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:55,217 - lava_callback - INFO - Received callback for node 69b0be4c9d1917581825285e INFO: 172.21.0.1:35584 - "POST /node/69b0be4c9d1917581825285e HTTP/1.0" 202 Accepted 2026-03-11 00:59:55,224 - lava_callback - INFO - Starting async job submit for node 69b0be4c9d1917581825285e 2026-03-11 00:59:55,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:55,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4c9d1917581825285e HTTP/1.1" 200 None 2026-03-11 00:59:55,266 - lava_callback - DEBUG - Processing node 69b0be4c9d1917581825285e: baseline-arm64-pengutronix 2026-03-11 00:59:55,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:55,429 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be079d19175818252773/log.txt.gz 2026-03-11 00:59:55,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:55,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:55,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:55,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4c9d1917581825285e/log.txt.gz 2026-03-11 00:59:55,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:55,923 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be079d19175818252773/lava-logs.yaml 2026-03-11 00:59:55,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:55,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:56,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:56,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4c9d1917581825285e/lava-logs.yaml 2026-03-11 00:59:56,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:56,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:56,154 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be079d19175818252773/lava_callback.json.gz 2026-03-11 00:59:56,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:56,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be079d19175818252773 HTTP/1.1" 200 None 2026-03-11 00:59:56,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:56,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:56,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:56,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be079d19175818252773 HTTP/1.1" 200 None 2026-03-11 00:59:56,257 - lava_callback - INFO - Completed processing callback for node 69b0be079d19175818252773 2026-03-11 00:59:56,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:56,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4c9d1917581825285e/lava_callback.json.gz 2026-03-11 00:59:56,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:56,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4c9d1917581825285e HTTP/1.1" 200 None 2026-03-11 00:59:56,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:56,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:56,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:56,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be4c9d1917581825285e HTTP/1.1" 200 None 2026-03-11 00:59:56,413 - lava_callback - INFO - Completed processing callback for node 69b0be4c9d1917581825285e INFO: 172.21.0.1:35586 - "GET / HTTP/1.0" 200 OK 2026-03-11 00:59:56,719 - lava_callback - INFO - Received callback for node 69b0be4e9d19175818252864 INFO: 172.21.0.1:35594 - "POST /node/69b0be4e9d19175818252864 HTTP/1.0" 202 Accepted 2026-03-11 00:59:56,726 - lava_callback - INFO - Starting async job submit for node 69b0be4e9d19175818252864 2026-03-11 00:59:56,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:56,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4e9d19175818252864 HTTP/1.1" 200 None 2026-03-11 00:59:56,764 - lava_callback - DEBUG - Processing node 69b0be4e9d19175818252864: baseline-arm64-foundriesio 2026-03-11 00:59:56,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:57,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:57,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b0be4e9d19175818252864/log.txt.gz 2026-03-11 00:59:57,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:57,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:57,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-11 00:59:57,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:57,398 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b0be4e9d19175818252864/lava-logs.yaml 2026-03-11 00:59:57,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:57,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:57,534 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b0be4e9d19175818252864/lava_callback.json.gz 2026-03-11 00:59:57,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:57,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4e9d19175818252864 HTTP/1.1" 200 None 2026-03-11 00:59:57,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:57,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:57,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:57,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be4e9d19175818252864 HTTP/1.1" 200 None 2026-03-11 00:59:57,614 - lava_callback - INFO - Completed processing callback for node 69b0be4e9d19175818252864 2026-03-11 00:59:58,773 - lava_callback - INFO - Received callback for node 69b0be0c9d1917581825277f INFO: 172.21.0.1:35610 - "POST /node/69b0be0c9d1917581825277f HTTP/1.0" 202 Accepted 2026-03-11 00:59:58,779 - lava_callback - INFO - Starting async job submit for node 69b0be0c9d1917581825277f 2026-03-11 00:59:58,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:58,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0c9d1917581825277f HTTP/1.1" 200 None 2026-03-11 00:59:58,857 - lava_callback - DEBUG - Processing node 69b0be0c9d1917581825277f: baseline-arm64 2026-03-11 00:59:59,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:59,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:59,201 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0c9d1917581825277f/log.txt.gz 2026-03-11 00:59:59,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:59,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:59,570 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0c9d1917581825277f/lava-logs.yaml 2026-03-11 00:59:59,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 00:59:59,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 00:59:59,751 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0c9d1917581825277f/lava_callback.json.gz 2026-03-11 00:59:59,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:59,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0c9d1917581825277f HTTP/1.1" 200 None 2026-03-11 00:59:59,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:59,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 00:59:59,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 00:59:59,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be0c9d1917581825277f HTTP/1.1" 200 None 2026-03-11 00:59:59,846 - lava_callback - INFO - Completed processing callback for node 69b0be0c9d1917581825277f 2026-03-11 01:00:00,905 - lava_callback - INFO - Received callback for node 69b0be189d191758182527b8 INFO: 172.21.0.1:45826 - "POST /node/69b0be189d191758182527b8 HTTP/1.0" 202 Accepted 2026-03-11 01:00:00,911 - lava_callback - INFO - Starting async job submit for node 69b0be189d191758182527b8 2026-03-11 01:00:00,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:00,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be189d191758182527b8 HTTP/1.1" 200 None 2026-03-11 01:00:00,949 - lava_callback - DEBUG - Processing node 69b0be189d191758182527b8: baseline-nfs-arm64 2026-03-11 01:00:01,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:01,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:01,507 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be189d191758182527b8/log.txt.gz 2026-03-11 01:00:01,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:01,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:01,939 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be189d191758182527b8/lava-logs.yaml 2026-03-11 01:00:01,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:02,055 - lava_callback - INFO - Received callback for node 69ae1b888ef4424e487b3b21 INFO: 172.21.0.1:45832 - "POST /node/69ae1b888ef4424e487b3b21 HTTP/1.0" 202 Accepted 2026-03-11 01:00:02,060 - lava_callback - INFO - Starting async job submit for node 69ae1b888ef4424e487b3b21 2026-03-11 01:00:02,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b888ef4424e487b3b21 HTTP/1.1" 200 None 2026-03-11 01:00:02,077 - lava_callback - DEBUG - Processing node 69ae1b888ef4424e487b3b21: baseline-arm-clabbe 2026-03-11 01:00:02,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1b888ef4424e487b3b21 2026-03-11 01:00:02,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1b888ef4424e487b3b21 HTTP/1.1" 200 None 2026-03-11 01:00:02,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:02,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be189d191758182527b8/lava_callback.json.gz 2026-03-11 01:00:02,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 01:00:02,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be189d191758182527b8 HTTP/1.1" 200 None 2026-03-11 01:00:02,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1b888ef4424e487b3b21 HTTP/1.1" 200 None 2026-03-11 01:00:02,161 - lava_callback - ERROR - Error processing callback for node 69ae1b888ef4424e487b3b21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:00:02,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:02,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be189d191758182527b8 HTTP/1.1" 200 None 2026-03-11 01:00:02,245 - lava_callback - INFO - Completed processing callback for node 69b0be189d191758182527b8 2026-03-11 01:00:02,890 - lava_callback - INFO - Received callback for node 69b0be239d191758182527d6 INFO: 172.21.0.1:45842 - "POST /node/69b0be239d191758182527d6 HTTP/1.0" 202 Accepted 2026-03-11 01:00:02,898 - lava_callback - INFO - Starting async job submit for node 69b0be239d191758182527d6 2026-03-11 01:00:02,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:02,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be239d191758182527d6 HTTP/1.1" 200 None 2026-03-11 01:00:02,940 - lava_callback - DEBUG - Processing node 69b0be239d191758182527d6: baseline-nfs-arm64 2026-03-11 01:00:03,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:03,406 - lava_callback - INFO - Received callback for node 69b0be4d9d19175818252861 INFO: 172.21.0.1:45856 - "POST /node/69b0be4d9d19175818252861 HTTP/1.0" 202 Accepted 2026-03-11 01:00:03,414 - lava_callback - INFO - Starting async job submit for node 69b0be4d9d19175818252861 2026-03-11 01:00:03,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:03,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:03,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be239d191758182527d6/log.txt.gz 2026-03-11 01:00:03,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4d9d19175818252861 HTTP/1.1" 200 None 2026-03-11 01:00:03,520 - lava_callback - DEBUG - Processing node 69b0be4d9d19175818252861: baseline-arm64-pengutronix 2026-03-11 01:00:03,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:03,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:04,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:04,033 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4d9d19175818252861/log.txt.gz 2026-03-11 01:00:04,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:04,120 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be239d191758182527d6/lava-logs.yaml 2026-03-11 01:00:04,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:04,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:04,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:04,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be239d191758182527d6/lava_callback.json.gz 2026-03-11 01:00:04,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:04,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:04,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4d9d19175818252861/lava-logs.yaml 2026-03-11 01:00:04,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:04,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be239d191758182527d6 HTTP/1.1" 200 None 2026-03-11 01:00:04,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:04,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:04,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:04,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be239d191758182527d6 HTTP/1.1" 200 None 2026-03-11 01:00:04,448 - lava_callback - INFO - Completed processing callback for node 69b0be239d191758182527d6 2026-03-11 01:00:04,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:04,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b0be4d9d19175818252861/lava_callback.json.gz 2026-03-11 01:00:04,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:04,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4d9d19175818252861 HTTP/1.1" 200 None 2026-03-11 01:00:04,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:04,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:04,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:04,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be4d9d19175818252861 HTTP/1.1" 200 None 2026-03-11 01:00:04,719 - lava_callback - INFO - Completed processing callback for node 69b0be4d9d19175818252861 INFO: 172.21.0.1:45872 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:00:09,222 - lava_callback - INFO - Received callback for node 69b0be129d1917581825278d INFO: 172.21.0.1:45878 - "POST /node/69b0be129d1917581825278d HTTP/1.0" 202 Accepted 2026-03-11 01:00:09,228 - lava_callback - INFO - Starting async job submit for node 69b0be129d1917581825278d 2026-03-11 01:00:09,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:09,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be129d1917581825278d HTTP/1.1" 200 None 2026-03-11 01:00:09,265 - lava_callback - DEBUG - Processing node 69b0be129d1917581825278d: baseline-arm64 2026-03-11 01:00:09,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:09,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:09,715 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be129d1917581825278d/log.txt.gz 2026-03-11 01:00:09,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:10,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:10,202 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be129d1917581825278d/lava-logs.yaml 2026-03-11 01:00:10,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:10,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:10,352 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be129d1917581825278d/lava_callback.json.gz 2026-03-11 01:00:10,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:10,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be129d1917581825278d HTTP/1.1" 200 None 2026-03-11 01:00:10,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:10,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:10,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:10,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be129d1917581825278d HTTP/1.1" 200 None 2026-03-11 01:00:10,443 - lava_callback - INFO - Completed processing callback for node 69b0be129d1917581825278d 2026-03-11 01:00:12,132 - lava_callback - INFO - Received callback for node 69b0be0f9d19175818252787 INFO: 172.21.0.1:52640 - "POST /node/69b0be0f9d19175818252787 HTTP/1.0" 202 Accepted 2026-03-11 01:00:12,139 - lava_callback - INFO - Starting async job submit for node 69b0be0f9d19175818252787 2026-03-11 01:00:12,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:12,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0f9d19175818252787 HTTP/1.1" 200 None 2026-03-11 01:00:12,178 - lava_callback - DEBUG - Processing node 69b0be0f9d19175818252787: baseline-arm64 2026-03-11 01:00:12,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:12,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:12,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0f9d19175818252787/log.txt.gz 2026-03-11 01:00:12,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:12,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:12,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0f9d19175818252787/lava-logs.yaml 2026-03-11 01:00:12,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:13,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:13,068 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0f9d19175818252787/lava_callback.json.gz 2026-03-11 01:00:13,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:13,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0f9d19175818252787 HTTP/1.1" 200 None 2026-03-11 01:00:13,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:13,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:13,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:13,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be0f9d19175818252787 HTTP/1.1" 200 None 2026-03-11 01:00:13,154 - lava_callback - INFO - Completed processing callback for node 69b0be0f9d19175818252787 INFO: 172.21.0.1:52646 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-11 01:00:15,944 - lava_callback - INFO - Received callback for node 69b0be339d1917581825281b INFO: 172.21.0.1:52648 - "POST /node/69b0be339d1917581825281b HTTP/1.0" 202 Accepted 2026-03-11 01:00:15,950 - lava_callback - INFO - Starting async job submit for node 69b0be339d1917581825281b 2026-03-11 01:00:15,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:15,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be339d1917581825281b HTTP/1.1" 200 None 2026-03-11 01:00:15,989 - lava_callback - DEBUG - Processing node 69b0be339d1917581825281b: baseline-arm64-broonie 2026-03-11 01:00:16,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:16,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:16,336 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be339d1917581825281b/log.txt.gz 2026-03-11 01:00:16,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:00:16,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:16,642 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be339d1917581825281b/lava-logs.yaml 2026-03-11 01:00:16,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:16,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:16,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be339d1917581825281b/lava_callback.json.gz 2026-03-11 01:00:16,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:16,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be339d1917581825281b HTTP/1.1" 200 None 2026-03-11 01:00:16,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:16,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:16,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be339d1917581825281b HTTP/1.1" 200 None 2026-03-11 01:00:16,860 - lava_callback - INFO - Completed processing callback for node 69b0be339d1917581825281b 2026-03-11 01:00:18,736 - lava_callback - INFO - Received callback for node 69b0be1c9d191758182527be INFO: 172.21.0.1:52658 - "POST /node/69b0be1c9d191758182527be HTTP/1.0" 202 Accepted 2026-03-11 01:00:18,743 - lava_callback - INFO - Starting async job submit for node 69b0be1c9d191758182527be 2026-03-11 01:00:18,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:18,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1c9d191758182527be HTTP/1.1" 200 None 2026-03-11 01:00:18,781 - lava_callback - DEBUG - Processing node 69b0be1c9d191758182527be: baseline-nfs-arm64 2026-03-11 01:00:19,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:19,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:19,403 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1c9d191758182527be/log.txt.gz 2026-03-11 01:00:19,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:19,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:19,909 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1c9d191758182527be/lava-logs.yaml 2026-03-11 01:00:19,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:20,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:20,084 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1c9d191758182527be/lava_callback.json.gz 2026-03-11 01:00:20,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:20,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1c9d191758182527be HTTP/1.1" 200 None 2026-03-11 01:00:20,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:20,131 - lava_callback - INFO - Received callback for node 69b0be1d9d191758182527c1 INFO: 172.21.0.1:41832 - "POST /node/69b0be1d9d191758182527c1 HTTP/1.0" 202 Accepted 2026-03-11 01:00:20,138 - lava_callback - INFO - Starting async job submit for node 69b0be1d9d191758182527c1 2026-03-11 01:00:20,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:20,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:20,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:20,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1d9d191758182527c1 HTTP/1.1" 200 None 2026-03-11 01:00:20,179 - lava_callback - DEBUG - Processing node 69b0be1d9d191758182527c1: baseline-nfs-arm64 2026-03-11 01:00:20,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be1c9d191758182527be HTTP/1.1" 200 None 2026-03-11 01:00:20,254 - lava_callback - INFO - Completed processing callback for node 69b0be1c9d191758182527be 2026-03-11 01:00:20,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:21,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:21,032 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1d9d191758182527c1/log.txt.gz 2026-03-11 01:00:21,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:21,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:21,628 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1d9d191758182527c1/lava-logs.yaml 2026-03-11 01:00:21,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:21,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:21,791 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1d9d191758182527c1/lava_callback.json.gz 2026-03-11 01:00:21,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:21,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1d9d191758182527c1 HTTP/1.1" 200 None 2026-03-11 01:00:21,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:21,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:21,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:21,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be1d9d191758182527c1 HTTP/1.1" 200 None 2026-03-11 01:00:21,886 - lava_callback - INFO - Completed processing callback for node 69b0be1d9d191758182527c1 2026-03-11 01:00:22,202 - lava_callback - INFO - Received callback for node 69b0be399d19175818252830 INFO: 172.21.0.1:41834 - "POST /node/69b0be399d19175818252830 HTTP/1.0" 202 Accepted 2026-03-11 01:00:22,210 - lava_callback - INFO - Starting async job submit for node 69b0be399d19175818252830 2026-03-11 01:00:22,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:22,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be399d19175818252830 HTTP/1.1" 200 None 2026-03-11 01:00:22,252 - lava_callback - DEBUG - Processing node 69b0be399d19175818252830: baseline-arm64-broonie 2026-03-11 01:00:22,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:22,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:22,664 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be399d19175818252830/log.txt.gz 2026-03-11 01:00:22,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:23,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:23,021 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be399d19175818252830/lava-logs.yaml 2026-03-11 01:00:23,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:23,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:23,163 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be399d19175818252830/lava_callback.json.gz 2026-03-11 01:00:23,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:23,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be399d19175818252830 HTTP/1.1" 200 None 2026-03-11 01:00:23,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:23,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:23,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:23,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be399d19175818252830 HTTP/1.1" 200 None 2026-03-11 01:00:23,257 - lava_callback - INFO - Completed processing callback for node 69b0be399d19175818252830 2026-03-11 01:00:23,641 - lava_callback - INFO - Received callback for node 69b0be409d1917581825283c INFO: 172.21.0.1:41838 - "POST /node/69b0be409d1917581825283c HTTP/1.0" 202 Accepted 2026-03-11 01:00:23,647 - lava_callback - INFO - Starting async job submit for node 69b0be409d1917581825283c 2026-03-11 01:00:23,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:23,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be409d1917581825283c HTTP/1.1" 200 None 2026-03-11 01:00:23,700 - lava_callback - DEBUG - Processing node 69b0be409d1917581825283c: baseline-arm64-broonie 2026-03-11 01:00:23,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:24,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:24,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be409d1917581825283c/log.txt.gz 2026-03-11 01:00:24,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:24,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:24,424 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be409d1917581825283c/lava-logs.yaml 2026-03-11 01:00:24,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:24,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:24,559 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be409d1917581825283c/lava_callback.json.gz 2026-03-11 01:00:24,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:24,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be409d1917581825283c HTTP/1.1" 200 None 2026-03-11 01:00:24,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:24,613 - lava_callback - INFO - Received callback for node 69b0be479d1917581825284e INFO: 172.21.0.1:41840 - "POST /node/69b0be479d1917581825284e HTTP/1.0" 202 Accepted 2026-03-11 01:00:24,620 - lava_callback - INFO - Starting async job submit for node 69b0be479d1917581825284e 2026-03-11 01:00:24,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:24,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:24,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:24,649 - lava_callback - INFO - Received callback for node 69b0bdff9d19175818252761 INFO: 172.21.0.1:41846 - "POST /node/69b0bdff9d19175818252761 HTTP/1.0" 202 Accepted 2026-03-11 01:00:24,660 - lava_callback - INFO - Starting async job submit for node 69b0bdff9d19175818252761 2026-03-11 01:00:24,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:24,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be479d1917581825284e HTTP/1.1" 200 None 2026-03-11 01:00:24,689 - lava_callback - DEBUG - Processing node 69b0be479d1917581825284e: baseline-arm64-clabbe 2026-03-11 01:00:24,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be409d1917581825283c HTTP/1.1" 200 None 2026-03-11 01:00:24,711 - lava_callback - INFO - Completed processing callback for node 69b0be409d1917581825283c 2026-03-11 01:00:24,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdff9d19175818252761 HTTP/1.1" 200 None 2026-03-11 01:00:24,856 - lava_callback - DEBUG - Processing node 69b0bdff9d19175818252761: baseline-arm64 2026-03-11 01:00:24,856 - lava_callback - INFO - Received callback for node 69b0be4f9d19175818252867 INFO: 172.21.0.1:41848 - "POST /node/69b0be4f9d19175818252867 HTTP/1.0" 202 Accepted 2026-03-11 01:00:24,927 - lava_callback - INFO - Starting async job submit for node 69b0be4f9d19175818252867 2026-03-11 01:00:25,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41858 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:00:25,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4f9d19175818252867 HTTP/1.1" 200 None 2026-03-11 01:00:25,337 - lava_callback - DEBUG - Processing node 69b0be4f9d19175818252867: baseline-arm64-foundriesio 2026-03-11 01:00:25,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:25,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:25,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:25,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:25,734 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdff9d19175818252761/log.txt.gz 2026-03-11 01:00:25,814 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b0be479d1917581825284e/log.txt.gz 2026-03-11 01:00:26,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:26,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:26,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:26,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:26,301 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b0be4f9d19175818252867/log.txt.gz 2026-03-11 01:00:26,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:26,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdff9d19175818252761/lava-logs.yaml 2026-03-11 01:00:26,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:26,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:26,501 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b0be479d1917581825284e/lava-logs.yaml INFO: 172.21.0.1:41870 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:00:26,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:26,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:26,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:26,764 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0bdff9d19175818252761/lava_callback.json.gz 2026-03-11 01:00:26,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:26,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:26,792 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b0be479d1917581825284e/lava_callback.json.gz 2026-03-11 01:00:26,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:26,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:26,819 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b0be4f9d19175818252867/lava-logs.yaml 2026-03-11 01:00:26,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be479d1917581825284e HTTP/1.1" 200 None 2026-03-11 01:00:26,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bdff9d19175818252761 HTTP/1.1" 200 None 2026-03-11 01:00:26,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:26,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:26,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:26,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:26,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:26,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:26,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:26,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be479d1917581825284e HTTP/1.1" 200 None 2026-03-11 01:00:26,889 - lava_callback - INFO - Completed processing callback for node 69b0be479d1917581825284e 2026-03-11 01:00:26,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bdff9d19175818252761 HTTP/1.1" 200 None 2026-03-11 01:00:26,914 - lava_callback - INFO - Completed processing callback for node 69b0bdff9d19175818252761 2026-03-11 01:00:27,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:27,086 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b0be4f9d19175818252867/lava_callback.json.gz 2026-03-11 01:00:27,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:27,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be4f9d19175818252867 HTTP/1.1" 200 None 2026-03-11 01:00:27,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:27,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:27,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:27,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be4f9d19175818252867 HTTP/1.1" 200 None 2026-03-11 01:00:27,222 - lava_callback - INFO - Completed processing callback for node 69b0be4f9d19175818252867 2026-03-11 01:00:27,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:27,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-11 01:00:32,485 - lava_callback - INFO - Received callback for node 69b0be279d191758182527dd INFO: 172.21.0.1:37374 - "POST /node/69b0be279d191758182527dd HTTP/1.0" 202 Accepted 2026-03-11 01:00:32,492 - lava_callback - INFO - Starting async job submit for node 69b0be279d191758182527dd 2026-03-11 01:00:32,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:32,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be279d191758182527dd HTTP/1.1" 200 None 2026-03-11 01:00:32,541 - lava_callback - DEBUG - Processing node 69b0be279d191758182527dd: baseline-nfs-arm64 2026-03-11 01:00:32,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:33,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:33,138 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be279d191758182527dd/log.txt.gz 2026-03-11 01:00:33,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:33,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:33,608 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be279d191758182527dd/lava-logs.yaml 2026-03-11 01:00:33,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:33,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:33,764 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be279d191758182527dd/lava_callback.json.gz 2026-03-11 01:00:33,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:33,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be279d191758182527dd HTTP/1.1" 200 None 2026-03-11 01:00:33,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:33,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:33,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:33,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be279d191758182527dd HTTP/1.1" 200 None 2026-03-11 01:00:33,891 - lava_callback - INFO - Completed processing callback for node 69b0be279d191758182527dd INFO: 172.21.0.1:37390 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:00:41,505 - lava_callback - INFO - Received callback for node 69b0be209d191758182527c7 INFO: 172.21.0.1:51388 - "POST /node/69b0be209d191758182527c7 HTTP/1.0" 202 Accepted 2026-03-11 01:00:41,511 - lava_callback - INFO - Starting async job submit for node 69b0be209d191758182527c7 2026-03-11 01:00:41,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:41,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be209d191758182527c7 HTTP/1.1" 200 None 2026-03-11 01:00:41,553 - lava_callback - DEBUG - Processing node 69b0be209d191758182527c7: baseline-nfs-arm64 2026-03-11 01:00:41,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:41,897 - lava_callback - INFO - Received callback for node 69ae1bb38ef4424e487b3b74 INFO: 172.21.0.1:51394 - "POST /node/69ae1bb38ef4424e487b3b74 HTTP/1.0" 202 Accepted 2026-03-11 01:00:41,903 - lava_callback - INFO - Starting async job submit for node 69ae1bb38ef4424e487b3b74 2026-03-11 01:00:41,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:41,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1bb38ef4424e487b3b74 HTTP/1.1" 200 None 2026-03-11 01:00:41,922 - lava_callback - DEBUG - Processing node 69ae1bb38ef4424e487b3b74: baseline-arm-clabbe 2026-03-11 01:00:41,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1bb38ef4424e487b3b74 2026-03-11 01:00:41,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:41,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1bb38ef4424e487b3b74 HTTP/1.1" 200 None 2026-03-11 01:00:41,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:41,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 01:00:41,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:41,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:41,979 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be209d191758182527c7/log.txt.gz 2026-03-11 01:00:41,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1bb38ef4424e487b3b74 HTTP/1.1" 200 None 2026-03-11 01:00:41,989 - lava_callback - ERROR - Error processing callback for node 69ae1bb38ef4424e487b3b74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:00:42,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:42,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:42,448 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be209d191758182527c7/lava-logs.yaml 2026-03-11 01:00:42,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:42,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:42,596 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be209d191758182527c7/lava_callback.json.gz 2026-03-11 01:00:42,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:42,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be209d191758182527c7 HTTP/1.1" 200 None 2026-03-11 01:00:42,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:42,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:42,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:42,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be209d191758182527c7 HTTP/1.1" 200 None 2026-03-11 01:00:42,677 - lava_callback - INFO - Completed processing callback for node 69b0be209d191758182527c7 INFO: 172.21.0.1:51398 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:00:50,459 - lava_callback - INFO - Received callback for node 69b0be159d191758182527b2 INFO: 172.21.0.1:41280 - "POST /node/69b0be159d191758182527b2 HTTP/1.0" 202 Accepted 2026-03-11 01:00:50,465 - lava_callback - INFO - Starting async job submit for node 69b0be159d191758182527b2 2026-03-11 01:00:50,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:50,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be159d191758182527b2 HTTP/1.1" 200 None 2026-03-11 01:00:50,512 - lava_callback - DEBUG - Processing node 69b0be159d191758182527b2: baseline-nfs-arm64 2026-03-11 01:00:50,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:51,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:51,015 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be159d191758182527b2/log.txt.gz 2026-03-11 01:00:51,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:51,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:51,439 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be159d191758182527b2/lava-logs.yaml 2026-03-11 01:00:51,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:51,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:51,573 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be159d191758182527b2/lava_callback.json.gz 2026-03-11 01:00:51,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:51,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be159d191758182527b2 HTTP/1.1" 200 None 2026-03-11 01:00:51,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:51,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:51,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:51,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be159d191758182527b2 HTTP/1.1" 200 None 2026-03-11 01:00:51,662 - lava_callback - INFO - Completed processing callback for node 69b0be159d191758182527b2 2026-03-11 01:00:55,916 - lava_callback - INFO - Received callback for node 69b0be429d1917581825283f INFO: 172.21.0.1:41294 - "POST /node/69b0be429d1917581825283f HTTP/1.0" 202 Accepted 2026-03-11 01:00:55,922 - lava_callback - INFO - Starting async job submit for node 69b0be429d1917581825283f 2026-03-11 01:00:55,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:55,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be429d1917581825283f HTTP/1.1" 200 None 2026-03-11 01:00:55,960 - lava_callback - DEBUG - Processing node 69b0be429d1917581825283f: baseline-arm64-broonie 2026-03-11 01:00:56,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:56,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:56,364 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be429d1917581825283f/log.txt.gz 2026-03-11 01:00:56,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:41302 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:00:56,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:56,711 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be429d1917581825283f/lava-logs.yaml 2026-03-11 01:00:56,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:56,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:56,866 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be429d1917581825283f/lava_callback.json.gz 2026-03-11 01:00:56,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:56,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be429d1917581825283f HTTP/1.1" 200 None 2026-03-11 01:00:56,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:56,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:00:56,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:56,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be429d1917581825283f HTTP/1.1" 200 None 2026-03-11 01:00:56,959 - lava_callback - INFO - Completed processing callback for node 69b0be429d1917581825283f 2026-03-11 01:00:57,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:57,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-11 01:00:58,835 - lava_callback - INFO - Received callback for node 69b0be0a9d19175818252779 INFO: 172.21.0.1:41318 - "POST /node/69b0be0a9d19175818252779 HTTP/1.0" 202 Accepted 2026-03-11 01:00:58,842 - lava_callback - INFO - Starting async job submit for node 69b0be0a9d19175818252779 2026-03-11 01:00:58,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:00:58,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0a9d19175818252779 HTTP/1.1" 200 None 2026-03-11 01:00:58,879 - lava_callback - DEBUG - Processing node 69b0be0a9d19175818252779: baseline-arm64 2026-03-11 01:00:59,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:00:59,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:00:59,534 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0a9d19175818252779/log.txt.gz 2026-03-11 01:00:59,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:00,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:00,040 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0a9d19175818252779/lava-logs.yaml 2026-03-11 01:01:00,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:00,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:00,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0a9d19175818252779/lava_callback.json.gz 2026-03-11 01:01:00,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:00,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0a9d19175818252779 HTTP/1.1" 200 None 2026-03-11 01:01:00,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:00,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:01:00,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:00,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be0a9d19175818252779 HTTP/1.1" 200 None 2026-03-11 01:01:00,359 - lava_callback - INFO - Completed processing callback for node 69b0be0a9d19175818252779 INFO: 172.21.0.1:40650 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:01:07,999 - lava_callback - INFO - Received callback for node 69b0be229d191758182527cd INFO: 172.21.0.1:40666 - "POST /node/69b0be229d191758182527cd HTTP/1.0" 202 Accepted 2026-03-11 01:01:08,010 - lava_callback - INFO - Starting async job submit for node 69b0be229d191758182527cd 2026-03-11 01:01:08,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:08,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be229d191758182527cd HTTP/1.1" 200 None 2026-03-11 01:01:08,057 - lava_callback - DEBUG - Processing node 69b0be229d191758182527cd: baseline-nfs-arm64 2026-03-11 01:01:08,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:08,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:08,703 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be229d191758182527cd/log.txt.gz 2026-03-11 01:01:08,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:09,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:09,183 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be229d191758182527cd/lava-logs.yaml 2026-03-11 01:01:09,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:09,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:09,387 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be229d191758182527cd/lava_callback.json.gz 2026-03-11 01:01:09,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:09,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be229d191758182527cd HTTP/1.1" 200 None 2026-03-11 01:01:09,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:09,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:01:09,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:09,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be229d191758182527cd HTTP/1.1" 200 None 2026-03-11 01:01:09,472 - lava_callback - INFO - Completed processing callback for node 69b0be229d191758182527cd INFO: 172.21.0.1:53254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53268 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:01:17,413 - lava_callback - INFO - Received callback for node 69b0be359d19175818252820 INFO: 172.21.0.1:53270 - "POST /node/69b0be359d19175818252820 HTTP/1.0" 202 Accepted 2026-03-11 01:01:17,419 - lava_callback - INFO - Starting async job submit for node 69b0be359d19175818252820 2026-03-11 01:01:17,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:17,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be359d19175818252820 HTTP/1.1" 200 None 2026-03-11 01:01:17,459 - lava_callback - DEBUG - Processing node 69b0be359d19175818252820: baseline-arm64-broonie 2026-03-11 01:01:17,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:17,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:17,808 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be359d19175818252820/log.txt.gz 2026-03-11 01:01:17,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:18,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:18,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be359d19175818252820/lava-logs.yaml 2026-03-11 01:01:18,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:01:18,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:01:18,325 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be359d19175818252820/lava_callback.json.gz 2026-03-11 01:01:18,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:18,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be359d19175818252820 HTTP/1.1" 200 None 2026-03-11 01:01:18,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:18,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:01:18,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be359d19175818252820 HTTP/1.1" 200 None 2026-03-11 01:01:18,411 - lava_callback - INFO - Completed processing callback for node 69b0be359d19175818252820 INFO: 172.21.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53182 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:01:27,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:01:27,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47556 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:02:00,069 - lava_callback - INFO - Received callback for node 69b0be0e9d19175818252783 INFO: 172.21.0.1:37056 - "POST /node/69b0be0e9d19175818252783 HTTP/1.0" 202 Accepted 2026-03-11 01:02:00,083 - lava_callback - INFO - Starting async job submit for node 69b0be0e9d19175818252783 2026-03-11 01:02:00,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:00,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0e9d19175818252783 HTTP/1.1" 200 None 2026-03-11 01:02:00,146 - lava_callback - DEBUG - Processing node 69b0be0e9d19175818252783: baseline-arm64 2026-03-11 01:02:02,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:02,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:02,903 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0e9d19175818252783/log.txt.gz 2026-03-11 01:02:03,491 - lava_callback - INFO - Received callback for node 69b0be049d1917581825276d INFO: 172.21.0.1:37066 - "POST /node/69b0be049d1917581825276d HTTP/1.0" 202 Accepted 2026-03-11 01:02:03,512 - lava_callback - INFO - Starting async job submit for node 69b0be049d1917581825276d 2026-03-11 01:02:03,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:03,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be049d1917581825276d HTTP/1.1" 200 None 2026-03-11 01:02:03,626 - lava_callback - DEBUG - Processing node 69b0be049d1917581825276d: baseline-arm64 2026-03-11 01:02:04,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:04,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:04,565 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be049d1917581825276d/log.txt.gz 2026-03-11 01:02:04,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:05,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:05,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:05,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be049d1917581825276d/lava-logs.yaml 2026-03-11 01:02:05,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:05,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:05,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0e9d19175818252783/lava-logs.yaml 2026-03-11 01:02:05,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:05,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:05,491 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be049d1917581825276d/lava_callback.json.gz 2026-03-11 01:02:05,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:05,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be049d1917581825276d HTTP/1.1" 200 None 2026-03-11 01:02:05,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:05,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:02:05,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:05,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be049d1917581825276d HTTP/1.1" 200 None 2026-03-11 01:02:05,597 - lava_callback - INFO - Completed processing callback for node 69b0be049d1917581825276d 2026-03-11 01:02:05,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:05,637 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be0e9d19175818252783/lava_callback.json.gz 2026-03-11 01:02:05,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:05,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be0e9d19175818252783 HTTP/1.1" 200 None 2026-03-11 01:02:05,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:05,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:02:05,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:05,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be0e9d19175818252783 HTTP/1.1" 200 None 2026-03-11 01:02:05,748 - lava_callback - INFO - Completed processing callback for node 69b0be0e9d19175818252783 INFO: 172.21.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:02:27,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:27,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 01:02:35,819 - lava_callback - INFO - Received callback for node 69b0be3b9d19175818252833 INFO: 172.21.0.1:36790 - "POST /node/69b0be3b9d19175818252833 HTTP/1.0" 202 Accepted 2026-03-11 01:02:35,830 - lava_callback - INFO - Starting async job submit for node 69b0be3b9d19175818252833 2026-03-11 01:02:35,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:35,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be3b9d19175818252833 HTTP/1.1" 200 None 2026-03-11 01:02:35,898 - lava_callback - DEBUG - Processing node 69b0be3b9d19175818252833: baseline-arm64-broonie 2026-03-11 01:02:36,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:02:36,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:36,710 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3b9d19175818252833/log.txt.gz 2026-03-11 01:02:36,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:37,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:37,446 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3b9d19175818252833/lava-logs.yaml 2026-03-11 01:02:37,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:02:37,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:02:37,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3b9d19175818252833/lava_callback.json.gz 2026-03-11 01:02:37,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:37,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be3b9d19175818252833 HTTP/1.1" 200 None 2026-03-11 01:02:37,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:37,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:02:37,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:37,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be3b9d19175818252833 HTTP/1.1" 200 None 2026-03-11 01:02:37,904 - lava_callback - INFO - Completed processing callback for node 69b0be3b9d19175818252833 INFO: 172.21.0.1:53200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41126 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:02:57,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:02:57,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33466 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:03:08,925 - lava_callback - INFO - Received callback for node 69b0be179d191758182527b5 INFO: 172.21.0.1:33482 - "POST /node/69b0be179d191758182527b5 HTTP/1.0" 202 Accepted 2026-03-11 01:03:08,935 - lava_callback - INFO - Starting async job submit for node 69b0be179d191758182527b5 2026-03-11 01:03:08,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:08,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be179d191758182527b5 HTTP/1.1" 200 None 2026-03-11 01:03:08,993 - lava_callback - DEBUG - Processing node 69b0be179d191758182527b5: baseline-nfs-arm64 2026-03-11 01:03:09,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:03:10,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:03:10,082 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be179d191758182527b5/log.txt.gz 2026-03-11 01:03:10,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:03:11,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:03:11,070 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be179d191758182527b5/lava-logs.yaml 2026-03-11 01:03:11,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:03:11,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:03:11,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be179d191758182527b5/lava_callback.json.gz 2026-03-11 01:03:11,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:11,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be179d191758182527b5 HTTP/1.1" 200 None 2026-03-11 01:03:11,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:11,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:03:11,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:11,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be179d191758182527b5 HTTP/1.1" 200 None 2026-03-11 01:03:11,581 - lava_callback - INFO - Completed processing callback for node 69b0be179d191758182527b5 INFO: 172.21.0.1:32980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:03:21,621 - lava_callback - INFO - Received callback for node 69b0be3e9d19175818252839 INFO: 172.21.0.1:46128 - "POST /node/69b0be3e9d19175818252839 HTTP/1.0" 202 Accepted 2026-03-11 01:03:21,632 - lava_callback - INFO - Starting async job submit for node 69b0be3e9d19175818252839 2026-03-11 01:03:21,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:21,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be3e9d19175818252839 HTTP/1.1" 200 None 2026-03-11 01:03:21,707 - lava_callback - DEBUG - Processing node 69b0be3e9d19175818252839: baseline-arm64-broonie 2026-03-11 01:03:22,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:03:22,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:03:22,996 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3e9d19175818252839/log.txt.gz 2026-03-11 01:03:23,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:03:24,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:03:24,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3e9d19175818252839/lava-logs.yaml 2026-03-11 01:03:24,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:03:24,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:03:24,580 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3e9d19175818252839/lava_callback.json.gz 2026-03-11 01:03:24,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:24,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be3e9d19175818252839 HTTP/1.1" 200 None 2026-03-11 01:03:24,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:24,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:03:24,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:24,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be3e9d19175818252839 HTTP/1.1" 200 None 2026-03-11 01:03:24,746 - lava_callback - INFO - Completed processing callback for node 69b0be3e9d19175818252839 INFO: 172.21.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46148 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:03:27,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:03:27,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47160 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:04:26,218 - lava_callback - INFO - Received callback for node 69b0b9d99d191758182523ec INFO: 172.21.0.1:47176 - "POST /node/69b0b9d99d191758182523ec HTTP/1.0" 202 Accepted 2026-03-11 01:04:26,225 - lava_callback - INFO - Starting async job submit for node 69b0b9d99d191758182523ec 2026-03-11 01:04:26,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:26,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d99d191758182523ec HTTP/1.1" 200 None 2026-03-11 01:04:26,295 - lava_callback - DEBUG - Processing node 69b0b9d99d191758182523ec: baseline-arm INFO: 172.21.0.1:47192 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:04:26,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:04:27,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:04:27,110 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d99d191758182523ec/log.txt.gz 2026-03-11 01:04:27,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:04:27,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:04:27,773 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d99d191758182523ec/lava-logs.yaml 2026-03-11 01:04:27,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:04:27,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:04:27,946 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0b9d99d191758182523ec/lava_callback.json.gz DEBUG: Setup failed for 69b0b9d99d191758182523ec. Transitting job node result: incomplete -> fail 2026-03-11 01:04:28,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:28,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b9d99d191758182523ec HTTP/1.1" 200 None 2026-03-11 01:04:28,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:28,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 01:04:28,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:28,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0b9d99d191758182523ec HTTP/1.1" 200 None 2026-03-11 01:04:28,248 - lava_callback - INFO - Completed processing callback for node 69b0b9d99d191758182523ec INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37280 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:04:51,200 - lava_callback - INFO - Received callback for node 69b0bf999d191758182529e9 INFO: 172.21.0.1:51802 - "POST /node/69b0bf999d191758182529e9 HTTP/1.0" 202 Accepted 2026-03-11 01:04:51,205 - lava_callback - INFO - Starting async job submit for node 69b0bf999d191758182529e9 2026-03-11 01:04:51,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:51,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bf999d191758182529e9 HTTP/1.1" 200 None 2026-03-11 01:04:51,235 - lava_callback - DEBUG - Processing node 69b0bf999d191758182529e9: baseline-nfs-arm64 2026-03-11 01:04:51,237 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:04:51,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:04:51,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:04:51,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0bf999d191758182529e9/lava-logs.yaml 2026-03-11 01:04:51,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:04:51,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:04:51,499 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0bf999d191758182529e9/lava_callback.json.gz 2026-03-11 01:04:51,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:51,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0bf999d191758182529e9 HTTP/1.1" 200 None 2026-03-11 01:04:51,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:51,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:04:51,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:51,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0bf999d191758182529e9 HTTP/1.1" 200 None 2026-03-11 01:04:51,572 - lava_callback - INFO - Completed processing callback for node 69b0bf999d191758182529e9 INFO: 172.21.0.1:51818 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:04:57,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:04:57,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46588 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:05:21,620 - lava_callback - INFO - Received callback for node 69b0be389d1917581825282c INFO: 172.21.0.1:44784 - "POST /node/69b0be389d1917581825282c HTTP/1.0" 202 Accepted 2026-03-11 01:05:21,627 - lava_callback - INFO - Starting async job submit for node 69b0be389d1917581825282c 2026-03-11 01:05:21,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:05:21,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be389d1917581825282c HTTP/1.1" 200 None 2026-03-11 01:05:21,681 - lava_callback - DEBUG - Processing node 69b0be389d1917581825282c: baseline-arm64-broonie 2026-03-11 01:05:22,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:05:22,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:05:22,362 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be389d1917581825282c/log.txt.gz 2026-03-11 01:05:22,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:05:22,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:05:22,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be389d1917581825282c/lava-logs.yaml 2026-03-11 01:05:22,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:05:23,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:05:23,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be389d1917581825282c/lava_callback.json.gz DEBUG: 69b0be389d1917581825282c Transitting job node result: pass -> incomplete 2026-03-11 01:05:23,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:05:23,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be389d1917581825282c HTTP/1.1" 200 None 2026-03-11 01:05:23,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:05:23,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:05:23,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:05:23,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be389d1917581825282c HTTP/1.1" 200 None 2026-03-11 01:05:23,199 - lava_callback - INFO - Completed processing callback for node 69b0be389d1917581825282c INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44802 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:05:27,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:05:27,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36734 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:06:04,518 - lava_callback - INFO - Received callback for node 69b0be149d191758182527af INFO: 172.21.0.1:56758 - "POST /node/69b0be149d191758182527af HTTP/1.0" 202 Accepted 2026-03-11 01:06:04,526 - lava_callback - INFO - Starting async job submit for node 69b0be149d191758182527af 2026-03-11 01:06:04,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:06:04,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be149d191758182527af HTTP/1.1" 200 None 2026-03-11 01:06:04,577 - lava_callback - DEBUG - Processing node 69b0be149d191758182527af: baseline-arm64 2026-03-11 01:06:04,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:06:05,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:06:05,110 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be149d191758182527af/log.txt.gz 2026-03-11 01:06:05,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:06:05,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:06:05,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be149d191758182527af/lava-logs.yaml 2026-03-11 01:06:05,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:06:05,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:06:05,722 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0be149d191758182527af/lava_callback.json.gz DEBUG: Setup failed for 69b0be149d191758182527af. Transitting job node result: incomplete -> fail 2026-03-11 01:06:05,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:06:05,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be149d191758182527af HTTP/1.1" 200 None 2026-03-11 01:06:05,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:06:05,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:06:05,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:06:05,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be149d191758182527af HTTP/1.1" 200 None 2026-03-11 01:06:05,963 - lava_callback - INFO - Completed processing callback for node 69b0be149d191758182527af INFO: 172.21.0.1:56760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33114 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:06:27,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:06:27,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46672 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:07:54,920 - lava_callback - INFO - Received callback for node 69b0c0059d19175818252a1a INFO: 172.21.0.1:54304 - "POST /node/69b0c0059d19175818252a1a HTTP/1.0" 202 Accepted 2026-03-11 01:07:54,927 - lava_callback - INFO - Starting async job submit for node 69b0c0059d19175818252a1a 2026-03-11 01:07:54,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:07:54,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0059d19175818252a1a HTTP/1.1" 200 None 2026-03-11 01:07:54,972 - lava_callback - DEBUG - Processing node 69b0c0059d19175818252a1a: baseline-arm64 2026-03-11 01:07:55,174 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:07:55,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:07:55,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:07:55,491 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c0059d19175818252a1a/lava-logs.yaml 2026-03-11 01:07:55,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:07:55,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:07:55,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c0059d19175818252a1a/lava_callback.json.gz 2026-03-11 01:07:55,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:07:55,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0059d19175818252a1a HTTP/1.1" 200 None 2026-03-11 01:07:55,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:07:55,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:07:55,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:07:55,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c0059d19175818252a1a HTTP/1.1" 200 None 2026-03-11 01:07:55,838 - lava_callback - INFO - Completed processing callback for node 69b0c0059d19175818252a1a INFO: 172.21.0.1:54318 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:07:57,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:07:57,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55010 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-11 01:08:15,785 - lava_callback - INFO - Received callback for node 69b0c0539d19175818252a2e INFO: 172.21.0.1:55018 - "POST /node/69b0c0539d19175818252a2e HTTP/1.0" 202 Accepted 2026-03-11 01:08:15,791 - lava_callback - INFO - Starting async job submit for node 69b0c0539d19175818252a2e 2026-03-11 01:08:15,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:08:15,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0539d19175818252a2e HTTP/1.1" 200 None 2026-03-11 01:08:15,837 - lava_callback - DEBUG - Processing node 69b0c0539d19175818252a2e: baseline-arm64 2026-03-11 01:08:15,972 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:08:16,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:08:16,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:08:16,203 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c0539d19175818252a2e/lava-logs.yaml 2026-03-11 01:08:16,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:08:16,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:08:16,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c0539d19175818252a2e/lava_callback.json.gz 2026-03-11 01:08:16,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:08:16,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0539d19175818252a2e HTTP/1.1" 200 None 2026-03-11 01:08:16,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:08:16,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:08:16,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:08:16,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c0539d19175818252a2e HTTP/1.1" 200 None 2026-03-11 01:08:16,488 - lava_callback - INFO - Completed processing callback for node 69b0c0539d19175818252a2e INFO: 172.21.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43998 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:08:27,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:08:27,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45354 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:09:50,706 - lava_callback - INFO - Received callback for node 69ae1dca8ef4424e487b3d36 INFO: 172.21.0.1:34220 - "POST /node/69ae1dca8ef4424e487b3d36 HTTP/1.0" 202 Accepted 2026-03-11 01:09:50,712 - lava_callback - INFO - Starting async job submit for node 69ae1dca8ef4424e487b3d36 2026-03-11 01:09:50,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:09:50,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1dca8ef4424e487b3d36 HTTP/1.1" 200 None 2026-03-11 01:09:50,728 - lava_callback - DEBUG - Processing node 69ae1dca8ef4424e487b3d36: baseline-arm64-baylibre 2026-03-11 01:09:50,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1dca8ef4424e487b3d36 2026-03-11 01:09:50,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:09:50,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1dca8ef4424e487b3d36 HTTP/1.1" 200 None 2026-03-11 01:09:50,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:09:50,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 01:09:50,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:09:50,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1dca8ef4424e487b3d36 HTTP/1.1" 200 None 2026-03-11 01:09:50,792 - lava_callback - ERROR - Error processing callback for node 69ae1dca8ef4424e487b3d36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:11:22,343 - lava_callback - INFO - Received callback for node 69ae1e258ef4424e487b3d53 INFO: 172.21.0.1:43684 - "POST /node/69ae1e258ef4424e487b3d53 HTTP/1.0" 202 Accepted 2026-03-11 01:11:22,349 - lava_callback - INFO - Starting async job submit for node 69ae1e258ef4424e487b3d53 2026-03-11 01:11:22,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:11:22,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1e258ef4424e487b3d53 HTTP/1.1" 200 None 2026-03-11 01:11:22,372 - lava_callback - DEBUG - Processing node 69ae1e258ef4424e487b3d53: baseline-arm64-clabbe 2026-03-11 01:11:22,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1e258ef4424e487b3d53 2026-03-11 01:11:22,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:11:22,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1e258ef4424e487b3d53 HTTP/1.1" 200 None 2026-03-11 01:11:22,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:11:22,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 01:11:22,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:11:22,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1e258ef4424e487b3d53 HTTP/1.1" 200 None 2026-03-11 01:11:22,429 - lava_callback - ERROR - Error processing callback for node 69ae1e258ef4424e487b3d53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40792 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:12:22,361 - lava_callback - INFO - Received callback for node 69ae1e6c8ef4424e487b3d7a INFO: 172.21.0.1:33994 - "POST /node/69ae1e6c8ef4424e487b3d7a HTTP/1.0" 202 Accepted 2026-03-11 01:12:22,366 - lava_callback - INFO - Starting async job submit for node 69ae1e6c8ef4424e487b3d7a 2026-03-11 01:12:22,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:22,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1e6c8ef4424e487b3d7a HTTP/1.1" 200 None 2026-03-11 01:12:22,383 - lava_callback - DEBUG - Processing node 69ae1e6c8ef4424e487b3d7a: baseline-arm-clabbe 2026-03-11 01:12:22,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1e6c8ef4424e487b3d7a 2026-03-11 01:12:22,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:22,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1e6c8ef4424e487b3d7a HTTP/1.1" 200 None 2026-03-11 01:12:22,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:22,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 01:12:22,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:22,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1e6c8ef4424e487b3d7a HTTP/1.1" 200 None 2026-03-11 01:12:22,440 - lava_callback - ERROR - Error processing callback for node 69ae1e6c8ef4424e487b3d7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45320 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:12:42,775 - lava_callback - INFO - Received callback for node 69b0be1a9d191758182527bb INFO: 172.21.0.1:57608 - "POST /node/69b0be1a9d191758182527bb HTTP/1.0" 202 Accepted 2026-03-11 01:12:42,783 - lava_callback - INFO - Starting async job submit for node 69b0be1a9d191758182527bb 2026-03-11 01:12:42,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:42,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1a9d191758182527bb HTTP/1.1" 200 None 2026-03-11 01:12:42,838 - lava_callback - DEBUG - Processing node 69b0be1a9d191758182527bb: baseline-nfs-arm64 2026-03-11 01:12:43,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:12:44,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:12:44,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1a9d191758182527bb/log.txt.gz 2026-03-11 01:12:44,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:12:45,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:12:45,049 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1a9d191758182527bb/lava-logs.yaml 2026-03-11 01:12:45,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:12:45,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:12:45,280 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0be1a9d191758182527bb/lava_callback.json.gz DEBUG: Setup failed for 69b0be1a9d191758182527bb. Transitting job node result: incomplete -> fail 2026-03-11 01:12:45,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:45,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be1a9d191758182527bb HTTP/1.1" 200 None 2026-03-11 01:12:45,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:45,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:12:45,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:45,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be1a9d191758182527bb HTTP/1.1" 200 None 2026-03-11 01:12:45,526 - lava_callback - INFO - Completed processing callback for node 69b0be1a9d191758182527bb INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56440 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:12:57,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:12:57,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 01:13:02,331 - lava_callback - INFO - Received callback for node 69ae1e978ef4424e487b3da4 INFO: 172.21.0.1:42230 - "POST /node/69ae1e978ef4424e487b3da4 HTTP/1.0" 202 Accepted 2026-03-11 01:13:02,336 - lava_callback - INFO - Starting async job submit for node 69ae1e978ef4424e487b3da4 2026-03-11 01:13:02,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:02,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1e978ef4424e487b3da4 HTTP/1.1" 200 None 2026-03-11 01:13:02,352 - lava_callback - DEBUG - Processing node 69ae1e978ef4424e487b3da4: baseline-arm-clabbe 2026-03-11 01:13:02,353 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1e978ef4424e487b3da4 2026-03-11 01:13:02,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:02,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1e978ef4424e487b3da4 HTTP/1.1" 200 None 2026-03-11 01:13:02,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:02,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 01:13:02,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:02,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1e978ef4424e487b3da4 HTTP/1.1" 200 None 2026-03-11 01:13:02,411 - lava_callback - ERROR - Error processing callback for node 69ae1e978ef4424e487b3da4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:13:28,470 - lava_callback - INFO - Received callback for node 69b0c13b9d19175818252a74 INFO: 172.21.0.1:50712 - "POST /node/69b0c13b9d19175818252a74 HTTP/1.0" 202 Accepted 2026-03-11 01:13:28,477 - lava_callback - INFO - Starting async job submit for node 69b0c13b9d19175818252a74 2026-03-11 01:13:28,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:28,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c13b9d19175818252a74 HTTP/1.1" 200 None 2026-03-11 01:13:28,513 - lava_callback - DEBUG - Processing node 69b0c13b9d19175818252a74: baseline-arm64 2026-03-11 01:13:29,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:13:29,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:13:29,210 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c13b9d19175818252a74/log.txt.gz 2026-03-11 01:13:29,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:13:29,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:13:29,766 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c13b9d19175818252a74/lava-logs.yaml 2026-03-11 01:13:29,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:13:29,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:13:29,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c13b9d19175818252a74/lava_callback.json.gz 2026-03-11 01:13:29,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:30,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c13b9d19175818252a74 HTTP/1.1" 200 None 2026-03-11 01:13:30,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:30,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:13:30,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:30,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c13b9d19175818252a74 HTTP/1.1" 200 None 2026-03-11 01:13:30,117 - lava_callback - INFO - Completed processing callback for node 69b0c13b9d19175818252a74 INFO: 172.21.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52174 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:13:57,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:13:57,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35890 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:14:51,127 - lava_callback - INFO - Received callback for node 69ae1f018ef4424e487b3eb6 INFO: 172.21.0.1:40280 - "POST /node/69ae1f018ef4424e487b3eb6 HTTP/1.0" 202 Accepted 2026-03-11 01:14:51,136 - lava_callback - INFO - Starting async job submit for node 69ae1f018ef4424e487b3eb6 2026-03-11 01:14:51,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f018ef4424e487b3eb6 HTTP/1.1" 200 None 2026-03-11 01:14:51,164 - lava_callback - DEBUG - Processing node 69ae1f018ef4424e487b3eb6: baseline-arm64-baylibre 2026-03-11 01:14:51,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f018ef4424e487b3eb6 2026-03-11 01:14:51,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f018ef4424e487b3eb6 HTTP/1.1" 200 None 2026-03-11 01:14:51,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:14:51,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f018ef4424e487b3eb6 HTTP/1.1" 200 None 2026-03-11 01:14:51,239 - lava_callback - ERROR - Error processing callback for node 69ae1f018ef4424e487b3eb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:14:51,803 - lava_callback - INFO - Received callback for node 69ae1f038ef4424e487b3eb9 INFO: 172.21.0.1:40288 - "POST /node/69ae1f038ef4424e487b3eb9 HTTP/1.0" 202 Accepted 2026-03-11 01:14:51,808 - lava_callback - INFO - Starting async job submit for node 69ae1f038ef4424e487b3eb9 2026-03-11 01:14:51,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f038ef4424e487b3eb9 HTTP/1.1" 200 None 2026-03-11 01:14:51,831 - lava_callback - DEBUG - Processing node 69ae1f038ef4424e487b3eb9: baseline-arm64-baylibre 2026-03-11 01:14:51,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f038ef4424e487b3eb9 2026-03-11 01:14:51,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f038ef4424e487b3eb9 HTTP/1.1" 200 None 2026-03-11 01:14:51,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:14:51,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:51,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f038ef4424e487b3eb9 HTTP/1.1" 200 None 2026-03-11 01:14:51,883 - lava_callback - ERROR - Error processing callback for node 69ae1f038ef4424e487b3eb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:14:52,490 - lava_callback - INFO - Received callback for node 69ae1f048ef4424e487b3ebc INFO: 172.21.0.1:40300 - "POST /node/69ae1f048ef4424e487b3ebc HTTP/1.0" 202 Accepted 2026-03-11 01:14:52,496 - lava_callback - INFO - Starting async job submit for node 69ae1f048ef4424e487b3ebc 2026-03-11 01:14:52,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:52,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f048ef4424e487b3ebc HTTP/1.1" 200 None 2026-03-11 01:14:52,527 - lava_callback - DEBUG - Processing node 69ae1f048ef4424e487b3ebc: baseline-arm64-baylibre 2026-03-11 01:14:52,527 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f048ef4424e487b3ebc 2026-03-11 01:14:52,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:52,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f048ef4424e487b3ebc HTTP/1.1" 200 None 2026-03-11 01:14:52,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:52,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:14:52,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:52,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f048ef4424e487b3ebc HTTP/1.1" 200 None 2026-03-11 01:14:52,594 - lava_callback - ERROR - Error processing callback for node 69ae1f048ef4424e487b3ebc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:14:53,182 - lava_callback - INFO - Received callback for node 69ae1f078ef4424e487b3ec3 INFO: 172.21.0.1:40306 - "POST /node/69ae1f078ef4424e487b3ec3 HTTP/1.0" 202 Accepted 2026-03-11 01:14:53,187 - lava_callback - INFO - Starting async job submit for node 69ae1f078ef4424e487b3ec3 2026-03-11 01:14:53,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:53,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f078ef4424e487b3ec3 HTTP/1.1" 200 None 2026-03-11 01:14:53,203 - lava_callback - DEBUG - Processing node 69ae1f078ef4424e487b3ec3: baseline-arm64-baylibre 2026-03-11 01:14:53,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f078ef4424e487b3ec3 2026-03-11 01:14:53,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:53,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f078ef4424e487b3ec3 HTTP/1.1" 200 None 2026-03-11 01:14:53,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:53,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:14:53,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:14:53,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f078ef4424e487b3ec3 HTTP/1.1" 200 None 2026-03-11 01:14:53,268 - lava_callback - ERROR - Error processing callback for node 69ae1f078ef4424e487b3ec3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35308 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:15:09,408 - lava_callback - INFO - Received callback for node 69b0c1fe9d19175818252ab1 INFO: 172.21.0.1:35316 - "POST /node/69b0c1fe9d19175818252ab1 HTTP/1.0" 202 Accepted 2026-03-11 01:15:09,414 - lava_callback - INFO - Starting async job submit for node 69b0c1fe9d19175818252ab1 2026-03-11 01:15:09,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:09,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c1fe9d19175818252ab1 HTTP/1.1" 200 None 2026-03-11 01:15:09,454 - lava_callback - DEBUG - Processing node 69b0c1fe9d19175818252ab1: baseline-nfs-arm64 2026-03-11 01:15:09,455 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:15:09,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:15:09,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:15:09,616 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0c1fe9d19175818252ab1/lava-logs.yaml 2026-03-11 01:15:09,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:15:09,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:15:09,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0c1fe9d19175818252ab1/lava_callback.json.gz 2026-03-11 01:15:09,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:09,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c1fe9d19175818252ab1 HTTP/1.1" 200 None 2026-03-11 01:15:09,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:09,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:15:09,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:09,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c1fe9d19175818252ab1 HTTP/1.1" 200 None 2026-03-11 01:15:09,830 - lava_callback - INFO - Completed processing callback for node 69b0c1fe9d19175818252ab1 2026-03-11 01:15:10,950 - lava_callback - INFO - Received callback for node 69ae1f098ef4424e487b3ec6 INFO: 172.21.0.1:42542 - "POST /node/69ae1f098ef4424e487b3ec6 HTTP/1.0" 202 Accepted 2026-03-11 01:15:10,955 - lava_callback - INFO - Starting async job submit for node 69ae1f098ef4424e487b3ec6 2026-03-11 01:15:10,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:10,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f098ef4424e487b3ec6 HTTP/1.1" 200 None 2026-03-11 01:15:10,972 - lava_callback - DEBUG - Processing node 69ae1f098ef4424e487b3ec6: baseline-arm64-baylibre 2026-03-11 01:15:10,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f098ef4424e487b3ec6 2026-03-11 01:15:10,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:10,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f098ef4424e487b3ec6 HTTP/1.1" 200 None 2026-03-11 01:15:10,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:11,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:11,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:11,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f098ef4424e487b3ec6 HTTP/1.1" 200 None 2026-03-11 01:15:11,068 - lava_callback - ERROR - Error processing callback for node 69ae1f098ef4424e487b3ec6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:15:11,613 - lava_callback - INFO - Received callback for node 69ae1f0b8ef4424e487b3ec9 INFO: 172.21.0.1:42546 - "POST /node/69ae1f0b8ef4424e487b3ec9 HTTP/1.0" 202 Accepted 2026-03-11 01:15:11,619 - lava_callback - INFO - Starting async job submit for node 69ae1f0b8ef4424e487b3ec9 2026-03-11 01:15:11,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:11,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f0b8ef4424e487b3ec9 HTTP/1.1" 200 None 2026-03-11 01:15:11,634 - lava_callback - DEBUG - Processing node 69ae1f0b8ef4424e487b3ec9: baseline-arm64-baylibre 2026-03-11 01:15:11,635 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f0b8ef4424e487b3ec9 2026-03-11 01:15:11,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:11,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f0b8ef4424e487b3ec9 HTTP/1.1" 200 None 2026-03-11 01:15:11,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:11,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:11,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:11,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f0b8ef4424e487b3ec9 HTTP/1.1" 200 None 2026-03-11 01:15:11,698 - lava_callback - ERROR - Error processing callback for node 69ae1f0b8ef4424e487b3ec9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42572 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:15:22,433 - lava_callback - INFO - Received callback for node 69ae1f1f8ef4424e487b3f17 INFO: 172.21.0.1:53480 - "POST /node/69ae1f1f8ef4424e487b3f17 HTTP/1.0" 202 Accepted 2026-03-11 01:15:22,438 - lava_callback - INFO - Starting async job submit for node 69ae1f1f8ef4424e487b3f17 2026-03-11 01:15:22,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:22,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f1f8ef4424e487b3f17 HTTP/1.1" 200 None 2026-03-11 01:15:22,454 - lava_callback - DEBUG - Processing node 69ae1f1f8ef4424e487b3f17: baseline-arm64-clabbe 2026-03-11 01:15:22,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f1f8ef4424e487b3f17 2026-03-11 01:15:22,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:22,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f1f8ef4424e487b3f17 HTTP/1.1" 200 None 2026-03-11 01:15:22,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:22,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:22,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:22,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f1f8ef4424e487b3f17 HTTP/1.1" 200 None 2026-03-11 01:15:22,515 - lava_callback - ERROR - Error processing callback for node 69ae1f1f8ef4424e487b3f17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:15:23,103 - lava_callback - INFO - Received callback for node 69ae1f218ef4424e487b3f1a INFO: 172.21.0.1:53496 - "POST /node/69ae1f218ef4424e487b3f1a HTTP/1.0" 202 Accepted 2026-03-11 01:15:23,108 - lava_callback - INFO - Starting async job submit for node 69ae1f218ef4424e487b3f1a 2026-03-11 01:15:23,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f218ef4424e487b3f1a HTTP/1.1" 200 None 2026-03-11 01:15:23,130 - lava_callback - DEBUG - Processing node 69ae1f218ef4424e487b3f1a: baseline-arm64-clabbe 2026-03-11 01:15:23,130 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f218ef4424e487b3f1a 2026-03-11 01:15:23,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f218ef4424e487b3f1a HTTP/1.1" 200 None 2026-03-11 01:15:23,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:23,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f218ef4424e487b3f1a HTTP/1.1" 200 None 2026-03-11 01:15:23,181 - lava_callback - ERROR - Error processing callback for node 69ae1f218ef4424e487b3f1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:15:23,779 - lava_callback - INFO - Received callback for node 69ae1f228ef4424e487b3f1d INFO: 172.21.0.1:53510 - "POST /node/69ae1f228ef4424e487b3f1d HTTP/1.0" 202 Accepted 2026-03-11 01:15:23,786 - lava_callback - INFO - Starting async job submit for node 69ae1f228ef4424e487b3f1d 2026-03-11 01:15:23,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f228ef4424e487b3f1d HTTP/1.1" 200 None 2026-03-11 01:15:23,801 - lava_callback - DEBUG - Processing node 69ae1f228ef4424e487b3f1d: baseline-arm64-clabbe 2026-03-11 01:15:23,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f228ef4424e487b3f1d 2026-03-11 01:15:23,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f228ef4424e487b3f1d HTTP/1.1" 200 None 2026-03-11 01:15:23,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:23,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:23,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f228ef4424e487b3f1d HTTP/1.1" 200 None 2026-03-11 01:15:23,871 - lava_callback - ERROR - Error processing callback for node 69ae1f228ef4424e487b3f1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:15:24,481 - lava_callback - INFO - Received callback for node 69ae1f228ef4424e487b3f20 INFO: 172.21.0.1:53526 - "POST /node/69ae1f228ef4424e487b3f20 HTTP/1.0" 202 Accepted 2026-03-11 01:15:24,487 - lava_callback - INFO - Starting async job submit for node 69ae1f228ef4424e487b3f20 2026-03-11 01:15:24,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:24,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f228ef4424e487b3f20 HTTP/1.1" 200 None 2026-03-11 01:15:24,502 - lava_callback - DEBUG - Processing node 69ae1f228ef4424e487b3f20: baseline-arm64-clabbe 2026-03-11 01:15:24,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f228ef4424e487b3f20 2026-03-11 01:15:24,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:24,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f228ef4424e487b3f20 HTTP/1.1" 200 None 2026-03-11 01:15:24,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:24,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:24,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:24,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f228ef4424e487b3f20 HTTP/1.1" 200 None 2026-03-11 01:15:24,575 - lava_callback - ERROR - Error processing callback for node 69ae1f228ef4424e487b3f20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-11 01:15:25,186 - lava_callback - INFO - Received callback for node 69ae1f268ef4424e487b3f2a INFO: 172.21.0.1:53538 - "POST /node/69ae1f268ef4424e487b3f2a HTTP/1.0" 202 Accepted 2026-03-11 01:15:25,192 - lava_callback - INFO - Starting async job submit for node 69ae1f268ef4424e487b3f2a 2026-03-11 01:15:25,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:25,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f268ef4424e487b3f2a HTTP/1.1" 200 None 2026-03-11 01:15:25,208 - lava_callback - DEBUG - Processing node 69ae1f268ef4424e487b3f2a: baseline-arm64-clabbe 2026-03-11 01:15:25,208 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f268ef4424e487b3f2a 2026-03-11 01:15:25,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:25,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f268ef4424e487b3f2a HTTP/1.1" 200 None 2026-03-11 01:15:25,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:25,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:25,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:25,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f268ef4424e487b3f2a HTTP/1.1" 200 None 2026-03-11 01:15:25,279 - lava_callback - ERROR - Error processing callback for node 69ae1f268ef4424e487b3f2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53544 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:15:25,937 - lava_callback - INFO - Received callback for node 69ae1f258ef4424e487b3f27 INFO: 172.21.0.1:53548 - "POST /node/69ae1f258ef4424e487b3f27 HTTP/1.0" 202 Accepted 2026-03-11 01:15:25,943 - lava_callback - INFO - Starting async job submit for node 69ae1f258ef4424e487b3f27 2026-03-11 01:15:25,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:25,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f258ef4424e487b3f27 HTTP/1.1" 200 None 2026-03-11 01:15:25,959 - lava_callback - DEBUG - Processing node 69ae1f258ef4424e487b3f27: baseline-arm64-clabbe 2026-03-11 01:15:25,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae1f258ef4424e487b3f27 2026-03-11 01:15:25,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:25,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1f258ef4424e487b3f27 HTTP/1.1" 200 None 2026-03-11 01:15:25,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:25,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 01:15:25,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:26,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1f258ef4424e487b3f27 HTTP/1.1" 200 None 2026-03-11 01:15:26,016 - lava_callback - ERROR - Error processing callback for node 69ae1f258ef4424e487b3f27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53550 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:15:27,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:27,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36442 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:15:56,556 - lava_callback - INFO - Received callback for node 69b0be209d191758182527c9 INFO: 172.21.0.1:36444 - "POST /node/69b0be209d191758182527c9 HTTP/1.0" 202 Accepted 2026-03-11 01:15:56,563 - lava_callback - INFO - Starting async job submit for node 69b0be209d191758182527c9 2026-03-11 01:15:56,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:56,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be209d191758182527c9 HTTP/1.1" 200 None 2026-03-11 01:15:56,643 - lava_callback - DEBUG - Processing node 69b0be209d191758182527c9: ltp-smoke 2026-03-11 01:15:57,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:15:57,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:15:57,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0be209d191758182527c9/log.txt.gz 2026-03-11 01:15:57,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:15:57,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:15:57,822 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0be209d191758182527c9/lava-logs.yaml 2026-03-11 01:15:57,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:15:58,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:15:58,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0be209d191758182527c9/lava_callback.json.gz 2026-03-11 01:15:58,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:58,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be209d191758182527c9 HTTP/1.1" 200 None 2026-03-11 01:15:58,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:58,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5059d19175818251fdc HTTP/1.1" 200 None 2026-03-11 01:15:58,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:15:58,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be209d191758182527c9 HTTP/1.1" 200 None 2026-03-11 01:15:58,312 - lava_callback - INFO - Completed processing callback for node 69b0be209d191758182527c9 INFO: 172.21.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:16:27,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:16:27,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57388 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:18:02,163 - lava_callback - INFO - Received callback for node 69b0c2559d19175818252ae2 INFO: 172.21.0.1:50556 - "POST /node/69b0c2559d19175818252ae2 HTTP/1.0" 202 Accepted 2026-03-11 01:18:02,169 - lava_callback - INFO - Starting async job submit for node 69b0c2559d19175818252ae2 2026-03-11 01:18:02,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:18:02,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c2559d19175818252ae2 HTTP/1.1" 200 None 2026-03-11 01:18:02,213 - lava_callback - DEBUG - Processing node 69b0c2559d19175818252ae2: baseline-arm64-broonie 2026-03-11 01:18:02,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:18:02,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:18:02,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0c2559d19175818252ae2/log.txt.gz 2026-03-11 01:18:02,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:18:03,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:18:03,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0c2559d19175818252ae2/lava-logs.yaml 2026-03-11 01:18:03,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:18:03,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:18:03,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0c2559d19175818252ae2/lava_callback.json.gz 2026-03-11 01:18:03,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:18:03,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c2559d19175818252ae2 HTTP/1.1" 200 None 2026-03-11 01:18:03,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:18:03,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:18:03,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:18:03,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c2559d19175818252ae2 HTTP/1.1" 200 None 2026-03-11 01:18:03,371 - lava_callback - INFO - Completed processing callback for node 69b0c2559d19175818252ae2 INFO: 172.21.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36992 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:18:27,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:18:27,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55274 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:19:17,953 - lava_callback - INFO - Received callback for node 69b0c2e39d19175818252afa INFO: 172.21.0.1:55276 - "POST /node/69b0c2e39d19175818252afa HTTP/1.0" 202 Accepted 2026-03-11 01:19:17,960 - lava_callback - INFO - Starting async job submit for node 69b0c2e39d19175818252afa 2026-03-11 01:19:17,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:19:18,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c2e39d19175818252afa HTTP/1.1" 200 None 2026-03-11 01:19:18,017 - lava_callback - DEBUG - Processing node 69b0c2e39d19175818252afa: baseline-arm64 2026-03-11 01:19:18,153 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:19:18,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:19:18,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:19:18,489 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c2e39d19175818252afa/lava-logs.yaml 2026-03-11 01:19:18,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:19:18,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:19:18,640 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c2e39d19175818252afa/lava_callback.json.gz 2026-03-11 01:19:18,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:19:18,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c2e39d19175818252afa HTTP/1.1" 200 None 2026-03-11 01:19:18,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:19:18,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:19:18,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:19:18,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c2e39d19175818252afa HTTP/1.1" 200 None 2026-03-11 01:19:18,799 - lava_callback - INFO - Completed processing callback for node 69b0c2e39d19175818252afa INFO: 172.21.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46828 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:19:27,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:19:27,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:20:36,097 - lava_callback - INFO - Received callback for node 69b0c3329d19175818252b03 INFO: 172.21.0.1:55770 - "POST /node/69b0c3329d19175818252b03 HTTP/1.0" 202 Accepted 2026-03-11 01:20:36,104 - lava_callback - INFO - Starting async job submit for node 69b0c3329d19175818252b03 2026-03-11 01:20:36,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:20:36,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3329d19175818252b03 HTTP/1.1" 200 None 2026-03-11 01:20:36,148 - lava_callback - DEBUG - Processing node 69b0c3329d19175818252b03: baseline-arm64 2026-03-11 01:20:36,286 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:20:36,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:20:36,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:20:36,583 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c3329d19175818252b03/lava-logs.yaml 2026-03-11 01:20:36,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:20:36,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:20:36,732 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c3329d19175818252b03/lava_callback.json.gz 2026-03-11 01:20:36,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:20:36,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c3329d19175818252b03 HTTP/1.1" 200 None 2026-03-11 01:20:36,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:20:36,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:20:36,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:20:36,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c3329d19175818252b03 HTTP/1.1" 200 None 2026-03-11 01:20:36,903 - lava_callback - INFO - Completed processing callback for node 69b0c3329d19175818252b03 INFO: 172.21.0.1:56020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41426 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:20:57,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:20:57,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51426 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:24:21,986 - lava_callback - INFO - Received callback for node 69b0c2989d19175818252aea INFO: 172.21.0.1:42134 - "POST /node/69b0c2989d19175818252aea HTTP/1.0" 202 Accepted 2026-03-11 01:24:21,998 - lava_callback - INFO - Starting async job submit for node 69b0c2989d19175818252aea 2026-03-11 01:24:22,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:22,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c2989d19175818252aea HTTP/1.1" 200 None 2026-03-11 01:24:22,152 - lava_callback - DEBUG - Processing node 69b0c2989d19175818252aea: baseline-arm64 2026-03-11 01:24:22,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:24:23,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:24:23,248 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c2989d19175818252aea/log.txt.gz 2026-03-11 01:24:23,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:24:24,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:24:24,130 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c2989d19175818252aea/lava-logs.yaml 2026-03-11 01:24:24,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:24:24,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:24:24,563 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c2989d19175818252aea/lava_callback.json.gz DEBUG: Setup failed for 69b0c2989d19175818252aea. Transitting job node result: incomplete -> fail 2026-03-11 01:24:24,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:24,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c2989d19175818252aea HTTP/1.1" 200 None 2026-03-11 01:24:24,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:24,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:24:24,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:24,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c2989d19175818252aea HTTP/1.1" 200 None 2026-03-11 01:24:24,907 - lava_callback - INFO - Completed processing callback for node 69b0c2989d19175818252aea INFO: 172.21.0.1:42138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42154 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:24:27,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:27,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 01:24:31,494 - lava_callback - INFO - Received callback for node 69ae21488ef4424e487b40bb INFO: 172.21.0.1:37518 - "POST /node/69ae21488ef4424e487b40bb HTTP/1.0" 202 Accepted 2026-03-11 01:24:31,506 - lava_callback - INFO - Starting async job submit for node 69ae21488ef4424e487b40bb 2026-03-11 01:24:31,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:31,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae21488ef4424e487b40bb HTTP/1.1" 200 None 2026-03-11 01:24:31,536 - lava_callback - DEBUG - Processing node 69ae21488ef4424e487b40bb: baseline-arm64-baylibre 2026-03-11 01:24:31,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae21488ef4424e487b40bb 2026-03-11 01:24:31,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:31,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae21488ef4424e487b40bb HTTP/1.1" 200 None 2026-03-11 01:24:31,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:31,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:24:31,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:24:31,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae21488ef4424e487b40bb HTTP/1.1" 200 None 2026-03-11 01:24:31,664 - lava_callback - ERROR - Error processing callback for node 69ae21488ef4424e487b40bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33622 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:25:51,592 - lava_callback - INFO - Received callback for node 69ae219a8ef4424e487b40da INFO: 172.21.0.1:39766 - "POST /node/69ae219a8ef4424e487b40da HTTP/1.0" 202 Accepted 2026-03-11 01:25:51,597 - lava_callback - INFO - Starting async job submit for node 69ae219a8ef4424e487b40da 2026-03-11 01:25:51,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:25:51,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae219a8ef4424e487b40da HTTP/1.1" 200 None 2026-03-11 01:25:51,617 - lava_callback - DEBUG - Processing node 69ae219a8ef4424e487b40da: baseline-arm64-baylibre 2026-03-11 01:25:51,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae219a8ef4424e487b40da 2026-03-11 01:25:51,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:25:51,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae219a8ef4424e487b40da HTTP/1.1" 200 None 2026-03-11 01:25:51,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:25:51,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:25:51,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:25:51,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae219a8ef4424e487b40da HTTP/1.1" 200 None 2026-03-11 01:25:51,672 - lava_callback - ERROR - Error processing callback for node 69ae219a8ef4424e487b40da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41162 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:27:11,643 - lava_callback - INFO - Received callback for node 69ae21e98ef4424e487b4101 INFO: 172.21.0.1:38788 - "POST /node/69ae21e98ef4424e487b4101 HTTP/1.0" 202 Accepted 2026-03-11 01:27:11,649 - lava_callback - INFO - Starting async job submit for node 69ae21e98ef4424e487b4101 2026-03-11 01:27:11,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:27:11,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae21e98ef4424e487b4101 HTTP/1.1" 200 None 2026-03-11 01:27:11,665 - lava_callback - DEBUG - Processing node 69ae21e98ef4424e487b4101: baseline-arm64-baylibre 2026-03-11 01:27:11,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae21e98ef4424e487b4101 2026-03-11 01:27:11,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:27:11,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae21e98ef4424e487b4101 HTTP/1.1" 200 None 2026-03-11 01:27:11,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:27:11,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:27:11,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:27:11,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae21e98ef4424e487b4101 HTTP/1.1" 200 None 2026-03-11 01:27:11,721 - lava_callback - ERROR - Error processing callback for node 69ae21e98ef4424e487b4101: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46880 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:28:00,767 - lava_callback - INFO - Received callback for node 69b0c1d29d19175818252a9f INFO: 172.21.0.1:42214 - "POST /node/69b0c1d29d19175818252a9f HTTP/1.0" 202 Accepted 2026-03-11 01:28:00,774 - lava_callback - INFO - Starting async job submit for node 69b0c1d29d19175818252a9f 2026-03-11 01:28:00,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:00,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c1d29d19175818252a9f HTTP/1.1" 200 None 2026-03-11 01:28:00,843 - lava_callback - DEBUG - Processing node 69b0c1d29d19175818252a9f: baseline-arm 2026-03-11 01:28:01,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:28:01,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:28:01,509 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0c1d29d19175818252a9f/log.txt.gz 2026-03-11 01:28:01,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:28:02,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:28:02,022 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0c1d29d19175818252a9f/lava-logs.yaml 2026-03-11 01:28:02,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:28:02,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:28:02,212 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0c1d29d19175818252a9f/lava_callback.json.gz DEBUG: Setup failed for 69b0c1d29d19175818252a9f. Transitting job node result: incomplete -> fail 2026-03-11 01:28:02,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:02,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c1d29d19175818252a9f HTTP/1.1" 200 None 2026-03-11 01:28:02,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:02,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 01:28:02,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:02,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c1d29d19175818252a9f HTTP/1.1" 200 None 2026-03-11 01:28:02,496 - lava_callback - INFO - Completed processing callback for node 69b0c1d29d19175818252a9f INFO: 172.21.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36758 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:28:27,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:27,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 01:28:31,688 - lava_callback - INFO - Received callback for node 69ae223b8ef4424e487b4125 INFO: 172.21.0.1:33568 - "POST /node/69ae223b8ef4424e487b4125 HTTP/1.0" 202 Accepted 2026-03-11 01:28:31,694 - lava_callback - INFO - Starting async job submit for node 69ae223b8ef4424e487b4125 2026-03-11 01:28:31,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:31,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae223b8ef4424e487b4125 HTTP/1.1" 200 None 2026-03-11 01:28:31,727 - lava_callback - DEBUG - Processing node 69ae223b8ef4424e487b4125: baseline-arm64-baylibre 2026-03-11 01:28:31,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae223b8ef4424e487b4125 2026-03-11 01:28:31,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:31,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae223b8ef4424e487b4125 HTTP/1.1" 200 None 2026-03-11 01:28:31,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:31,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:28:31,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:31,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae223b8ef4424e487b4125 HTTP/1.1" 200 None 2026-03-11 01:28:31,813 - lava_callback - ERROR - Error processing callback for node 69ae223b8ef4424e487b4125: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33572 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:28:44,382 - lava_callback - INFO - Received callback for node 69b0c50f9d19175818252b53 INFO: 172.21.0.1:49804 - "POST /node/69b0c50f9d19175818252b53 HTTP/1.0" 202 Accepted 2026-03-11 01:28:44,388 - lava_callback - INFO - Starting async job submit for node 69b0c50f9d19175818252b53 2026-03-11 01:28:44,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:44,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c50f9d19175818252b53 HTTP/1.1" 200 None 2026-03-11 01:28:44,424 - lava_callback - DEBUG - Processing node 69b0c50f9d19175818252b53: baseline-arm64 2026-03-11 01:28:44,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:28:45,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:28:45,053 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c50f9d19175818252b53/log.txt.gz 2026-03-11 01:28:45,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:28:45,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:28:45,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c50f9d19175818252b53/lava-logs.yaml 2026-03-11 01:28:45,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:28:45,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:28:45,678 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c50f9d19175818252b53/lava_callback.json.gz 2026-03-11 01:28:45,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:45,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c50f9d19175818252b53 HTTP/1.1" 200 None 2026-03-11 01:28:45,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:45,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:28:45,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:45,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c50f9d19175818252b53 HTTP/1.1" 200 None 2026-03-11 01:28:45,761 - lava_callback - INFO - Completed processing callback for node 69b0c50f9d19175818252b53 INFO: 172.21.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:28:57,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:28:57,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-11 01:29:03,671 - lava_callback - INFO - Received callback for node 69b0c0d19d19175818252a51 INFO: 172.21.0.1:55152 - "POST /node/69b0c0d19d19175818252a51 HTTP/1.0" 202 Accepted 2026-03-11 01:29:03,717 - lava_callback - INFO - Starting async job submit for node 69b0c0d19d19175818252a51 2026-03-11 01:29:03,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0d19d19175818252a51 HTTP/1.1" 200 None 2026-03-11 01:29:03,756 - lava_callback - DEBUG - Processing node 69b0c0d19d19175818252a51: baseline-arm-pengutronix 2026-03-11 01:29:03,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:29:04,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:29:04,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0c0d19d19175818252a51/log.txt.gz 2026-03-11 01:29:04,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:29:04,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:29:04,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0c0d19d19175818252a51/lava-logs.yaml 2026-03-11 01:29:04,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:29:04,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:29:04,398 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0c0d19d19175818252a51/lava_callback.json.gz DEBUG: Setup failed for 69b0c0d19d19175818252a51. Transitting job node result: incomplete -> fail 2026-03-11 01:29:04,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:04,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c0d19d19175818252a51 HTTP/1.1" 200 None 2026-03-11 01:29:04,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:04,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 01:29:04,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:04,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c0d19d19175818252a51 HTTP/1.1" 200 None 2026-03-11 01:29:04,554 - lava_callback - INFO - Completed processing callback for node 69b0c0d19d19175818252a51 INFO: 172.21.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48504 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:29:27,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:27,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47076 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:29:51,772 - lava_callback - INFO - Received callback for node 69ae228b8ef4424e487b4147 INFO: 172.21.0.1:54020 - "POST /node/69ae228b8ef4424e487b4147 HTTP/1.0" 202 Accepted 2026-03-11 01:29:51,777 - lava_callback - INFO - Starting async job submit for node 69ae228b8ef4424e487b4147 2026-03-11 01:29:51,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:51,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae228b8ef4424e487b4147 HTTP/1.1" 200 None 2026-03-11 01:29:51,801 - lava_callback - DEBUG - Processing node 69ae228b8ef4424e487b4147: baseline-arm64-baylibre 2026-03-11 01:29:51,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae228b8ef4424e487b4147 2026-03-11 01:29:51,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:51,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae228b8ef4424e487b4147 HTTP/1.1" 200 None 2026-03-11 01:29:51,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:51,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:29:51,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:29:51,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae228b8ef4424e487b4147 HTTP/1.1" 200 None 2026-03-11 01:29:51,859 - lava_callback - ERROR - Error processing callback for node 69ae228b8ef4424e487b4147: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50552 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:31:31,883 - lava_callback - INFO - Received callback for node 69ae22e08ef4424e487b4178 INFO: 172.21.0.1:44540 - "POST /node/69ae22e08ef4424e487b4178 HTTP/1.0" 202 Accepted 2026-03-11 01:31:31,889 - lava_callback - INFO - Starting async job submit for node 69ae22e08ef4424e487b4178 2026-03-11 01:31:31,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:31:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae22e08ef4424e487b4178 HTTP/1.1" 200 None 2026-03-11 01:31:31,906 - lava_callback - DEBUG - Processing node 69ae22e08ef4424e487b4178: baseline-arm64-baylibre 2026-03-11 01:31:31,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae22e08ef4424e487b4178 2026-03-11 01:31:31,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:31:31,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae22e08ef4424e487b4178 HTTP/1.1" 200 None 2026-03-11 01:31:31,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:31:31,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:31:31,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:31:31,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae22e08ef4424e487b4178 HTTP/1.1" 200 None 2026-03-11 01:31:31,960 - lava_callback - ERROR - Error processing callback for node 69ae22e08ef4424e487b4178: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET /node/699448525ae459b9df3dad23%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:56044 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:34:03,292 - lava_callback - INFO - Received callback for node 69ae23848ef4424e487b41dd INFO: 172.21.0.1:51092 - "POST /node/69ae23848ef4424e487b41dd HTTP/1.0" 202 Accepted 2026-03-11 01:34:03,298 - lava_callback - INFO - Starting async job submit for node 69ae23848ef4424e487b41dd 2026-03-11 01:34:03,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:03,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae23848ef4424e487b41dd HTTP/1.1" 200 None 2026-03-11 01:34:03,314 - lava_callback - DEBUG - Processing node 69ae23848ef4424e487b41dd: baseline-arm64-clabbe 2026-03-11 01:34:03,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae23848ef4424e487b41dd 2026-03-11 01:34:03,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:03,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae23848ef4424e487b41dd HTTP/1.1" 200 None 2026-03-11 01:34:03,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:03,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:34:03,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:03,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae23848ef4424e487b41dd HTTP/1.1" 200 None 2026-03-11 01:34:03,381 - lava_callback - ERROR - Error processing callback for node 69ae23848ef4424e487b41dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:34:50,171 - lava_callback - INFO - Received callback for node 69b0c6a29d19175818252b97 INFO: 172.21.0.1:46068 - "POST /node/69b0c6a29d19175818252b97 HTTP/1.0" 202 Accepted 2026-03-11 01:34:50,177 - lava_callback - INFO - Starting async job submit for node 69b0c6a29d19175818252b97 2026-03-11 01:34:50,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:50,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6a29d19175818252b97 HTTP/1.1" 200 None 2026-03-11 01:34:50,194 - lava_callback - DEBUG - Processing node 69b0c6a29d19175818252b97: baseline-nfs-arm64 2026-03-11 01:34:50,195 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:34:50,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:34:50,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:34:50,344 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0c6a29d19175818252b97/lava-logs.yaml 2026-03-11 01:34:50,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:34:50,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:34:50,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0c6a29d19175818252b97/lava_callback.json.gz 2026-03-11 01:34:50,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:50,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c6a29d19175818252b97 HTTP/1.1" 200 None 2026-03-11 01:34:50,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:50,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:34:50,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:50,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c6a29d19175818252b97 HTTP/1.1" 200 None 2026-03-11 01:34:50,556 - lava_callback - INFO - Completed processing callback for node 69b0c6a29d19175818252b97 INFO: 172.21.0.1:46072 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:34:57,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:34:57,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:35:23,325 - lava_callback - INFO - Received callback for node 69ae23d58ef4424e487b420d INFO: 172.21.0.1:33574 - "POST /node/69ae23d58ef4424e487b420d HTTP/1.0" 202 Accepted 2026-03-11 01:35:23,331 - lava_callback - INFO - Starting async job submit for node 69ae23d58ef4424e487b420d 2026-03-11 01:35:23,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:35:23,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae23d58ef4424e487b420d HTTP/1.1" 200 None 2026-03-11 01:35:23,348 - lava_callback - DEBUG - Processing node 69ae23d58ef4424e487b420d: baseline-arm64-clabbe 2026-03-11 01:35:23,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae23d58ef4424e487b420d 2026-03-11 01:35:23,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:35:23,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae23d58ef4424e487b420d HTTP/1.1" 200 None 2026-03-11 01:35:23,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:35:23,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 01:35:23,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:35:23,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae23d58ef4424e487b420d HTTP/1.1" 200 None 2026-03-11 01:35:23,502 - lava_callback - ERROR - Error processing callback for node 69ae23d58ef4424e487b420d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55322 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:44:23,372 - lava_callback - INFO - Received callback for node 69b0c44f9d19175818252b33 INFO: 172.21.0.1:44810 - "POST /node/69b0c44f9d19175818252b33 HTTP/1.0" 202 Accepted 2026-03-11 01:44:23,380 - lava_callback - INFO - Starting async job submit for node 69b0c44f9d19175818252b33 2026-03-11 01:44:23,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:44:23,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c44f9d19175818252b33 HTTP/1.1" 200 None 2026-03-11 01:44:23,441 - lava_callback - DEBUG - Processing node 69b0c44f9d19175818252b33: baseline-nfs-arm64 2026-03-11 01:44:24,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:44:24,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:44:24,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0c44f9d19175818252b33/log.txt.gz 2026-03-11 01:44:25,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:44:25,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:44:25,611 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0c44f9d19175818252b33/lava-logs.yaml 2026-03-11 01:44:25,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:44:25,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:44:25,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0c44f9d19175818252b33/lava_callback.json.gz DEBUG: Setup failed for 69b0c44f9d19175818252b33. Transitting job node result: incomplete -> fail 2026-03-11 01:44:25,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:44826 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:44:25,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c44f9d19175818252b33 HTTP/1.1" 200 None 2026-03-11 01:44:25,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:44:25,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:44:25,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:44:25,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c44f9d19175818252b33 HTTP/1.1" 200 None 2026-03-11 01:44:26,095 - lava_callback - INFO - Completed processing callback for node 69b0c44f9d19175818252b33 INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:44:27,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:44:27,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:47:12,986 - lava_callback - INFO - Received callback for node 69ae269d8ef4424e487b439e INFO: 172.21.0.1:46540 - "POST /node/69ae269d8ef4424e487b439e HTTP/1.0" 202 Accepted 2026-03-11 01:47:12,993 - lava_callback - INFO - Starting async job submit for node 69ae269d8ef4424e487b439e 2026-03-11 01:47:12,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:13,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae269d8ef4424e487b439e HTTP/1.1" 200 None 2026-03-11 01:47:13,020 - lava_callback - DEBUG - Processing node 69ae269d8ef4424e487b439e: baseline-arm64-baylibre 2026-03-11 01:47:13,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae269d8ef4424e487b439e 2026-03-11 01:47:13,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:13,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae269d8ef4424e487b439e HTTP/1.1" 200 None 2026-03-11 01:47:13,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:13,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-11 01:47:13,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:13,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae269d8ef4424e487b439e HTTP/1.1" 200 None 2026-03-11 01:47:13,071 - lava_callback - ERROR - Error processing callback for node 69ae269d8ef4424e487b439e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42216 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:47:31,243 - lava_callback - INFO - Received callback for node 69b0c9719d19175818252bee INFO: 172.21.0.1:46008 - "POST /node/69b0c9719d19175818252bee HTTP/1.0" 202 Accepted 2026-03-11 01:47:31,250 - lava_callback - INFO - Starting async job submit for node 69b0c9719d19175818252bee 2026-03-11 01:47:31,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:31,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9719d19175818252bee HTTP/1.1" 200 None 2026-03-11 01:47:31,320 - lava_callback - DEBUG - Processing node 69b0c9719d19175818252bee: baseline-arm64 2026-03-11 01:47:31,523 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:47:31,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:47:31,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:47:31,848 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c9719d19175818252bee/lava-logs.yaml 2026-03-11 01:47:31,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:47:31,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:47:31,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c9719d19175818252bee/lava_callback.json.gz 2026-03-11 01:47:32,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:32,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9719d19175818252bee HTTP/1.1" 200 None 2026-03-11 01:47:32,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:32,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:47:32,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:32,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c9719d19175818252bee HTTP/1.1" 200 None 2026-03-11 01:47:32,145 - lava_callback - INFO - Completed processing callback for node 69b0c9719d19175818252bee INFO: 172.21.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57100 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:47:57,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:47:57,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41886 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:48:39,319 - lava_callback - INFO - Received callback for node 69b0c9c09d19175818252bf9 INFO: 172.21.0.1:41892 - "POST /node/69b0c9c09d19175818252bf9 HTTP/1.0" 202 Accepted 2026-03-11 01:48:39,326 - lava_callback - INFO - Starting async job submit for node 69b0c9c09d19175818252bf9 2026-03-11 01:48:39,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:39,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9c09d19175818252bf9 HTTP/1.1" 200 None 2026-03-11 01:48:39,369 - lava_callback - DEBUG - Processing node 69b0c9c09d19175818252bf9: baseline-arm64 2026-03-11 01:48:39,525 - lava_callback - WARNING - Failed to upload log 2026-03-11 01:48:39,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:48:39,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:48:39,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c9c09d19175818252bf9/lava-logs.yaml 2026-03-11 01:48:39,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:48:39,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:48:39,955 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0c9c09d19175818252bf9/lava_callback.json.gz 2026-03-11 01:48:39,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:40,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c9c09d19175818252bf9 HTTP/1.1" 200 None 2026-03-11 01:48:40,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:40,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:48:40,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:40,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c9c09d19175818252bf9 HTTP/1.1" 200 None 2026-03-11 01:48:40,136 - lava_callback - INFO - Completed processing callback for node 69b0c9c09d19175818252bf9 INFO: 172.21.0.1:57028 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:48:52,801 - lava_callback - INFO - Received callback for node 69ae26f78ef4424e487b43b2 INFO: 172.21.0.1:36114 - "POST /node/69ae26f78ef4424e487b43b2 HTTP/1.0" 202 Accepted 2026-03-11 01:48:52,807 - lava_callback - INFO - Starting async job submit for node 69ae26f78ef4424e487b43b2 2026-03-11 01:48:52,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:52,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae26f78ef4424e487b43b2 HTTP/1.1" 200 None 2026-03-11 01:48:52,822 - lava_callback - DEBUG - Processing node 69ae26f78ef4424e487b43b2: baseline-arm64-baylibre 2026-03-11 01:48:52,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae26f78ef4424e487b43b2 2026-03-11 01:48:52,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:52,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae26f78ef4424e487b43b2 HTTP/1.1" 200 None 2026-03-11 01:48:52,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:52,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-11 01:48:52,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:52,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae26f78ef4424e487b43b2 HTTP/1.1" 200 None 2026-03-11 01:48:52,882 - lava_callback - ERROR - Error processing callback for node 69ae26f78ef4424e487b43b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36120 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:48:57,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:48:57,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:50:12,878 - lava_callback - INFO - Received callback for node 69ae274a8ef4424e487b43cd INFO: 172.21.0.1:51068 - "POST /node/69ae274a8ef4424e487b43cd HTTP/1.0" 202 Accepted 2026-03-11 01:50:12,884 - lava_callback - INFO - Starting async job submit for node 69ae274a8ef4424e487b43cd 2026-03-11 01:50:12,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:50:12,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae274a8ef4424e487b43cd HTTP/1.1" 200 None 2026-03-11 01:50:12,899 - lava_callback - DEBUG - Processing node 69ae274a8ef4424e487b43cd: baseline-arm64-baylibre 2026-03-11 01:50:12,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae274a8ef4424e487b43cd 2026-03-11 01:50:12,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:50:12,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae274a8ef4424e487b43cd HTTP/1.1" 200 None 2026-03-11 01:50:12,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:50:12,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-11 01:50:12,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:50:12,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae274a8ef4424e487b43cd HTTP/1.1" 200 None 2026-03-11 01:50:12,961 - lava_callback - ERROR - Error processing callback for node 69ae274a8ef4424e487b43cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38064 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:54:33,148 - lava_callback - INFO - Received callback for node 69ae28458ef4424e487b4422 INFO: 172.21.0.1:50508 - "POST /node/69ae28458ef4424e487b4422 HTTP/1.0" 202 Accepted 2026-03-11 01:54:33,155 - lava_callback - INFO - Starting async job submit for node 69ae28458ef4424e487b4422 2026-03-11 01:54:33,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:54:33,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae28458ef4424e487b4422 HTTP/1.1" 200 None 2026-03-11 01:54:33,188 - lava_callback - DEBUG - Processing node 69ae28458ef4424e487b4422: baseline-arm64-baylibre 2026-03-11 01:54:33,188 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae28458ef4424e487b4422 2026-03-11 01:54:33,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:54:33,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae28458ef4424e487b4422 HTTP/1.1" 200 None 2026-03-11 01:54:33,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:54:33,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-11 01:54:33,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:54:33,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae28458ef4424e487b4422 HTTP/1.1" 200 None 2026-03-11 01:54:33,246 - lava_callback - ERROR - Error processing callback for node 69ae28458ef4424e487b4422: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38238 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:55:53,218 - lava_callback - INFO - Received callback for node 69ae28958ef4424e487b443b INFO: 172.21.0.1:33932 - "POST /node/69ae28958ef4424e487b443b HTTP/1.0" 202 Accepted 2026-03-11 01:55:53,223 - lava_callback - INFO - Starting async job submit for node 69ae28958ef4424e487b443b 2026-03-11 01:55:53,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:55:53,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae28958ef4424e487b443b HTTP/1.1" 200 None 2026-03-11 01:55:53,253 - lava_callback - DEBUG - Processing node 69ae28958ef4424e487b443b: baseline-arm64-baylibre 2026-03-11 01:55:53,253 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae28958ef4424e487b443b 2026-03-11 01:55:53,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:55:53,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae28958ef4424e487b443b HTTP/1.1" 200 None 2026-03-11 01:55:53,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:55:53,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-11 01:55:53,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:55:53,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae28958ef4424e487b443b HTTP/1.1" 200 None 2026-03-11 01:55:53,313 - lava_callback - ERROR - Error processing callback for node 69ae28958ef4424e487b443b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:56:33,637 - lava_callback - INFO - Received callback for node 69b0ca159d19175818252c07 INFO: 172.21.0.1:47562 - "POST /node/69b0ca159d19175818252c07 HTTP/1.0" 202 Accepted 2026-03-11 01:56:33,644 - lava_callback - INFO - Starting async job submit for node 69b0ca159d19175818252c07 2026-03-11 01:56:33,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:56:33,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ca159d19175818252c07 HTTP/1.1" 200 None 2026-03-11 01:56:33,695 - lava_callback - DEBUG - Processing node 69b0ca159d19175818252c07: baseline-arm64 2026-03-11 01:56:34,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:56:34,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:56:34,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0ca159d19175818252c07/log.txt.gz 2026-03-11 01:56:34,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:56:34,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:56:34,659 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0ca159d19175818252c07/lava-logs.yaml 2026-03-11 01:56:34,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 01:56:34,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 01:56:34,829 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0ca159d19175818252c07/lava_callback.json.gz DEBUG: Setup failed for 69b0ca159d19175818252c07. Transitting job node result: incomplete -> fail 2026-03-11 01:56:34,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:56:34,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0ca159d19175818252c07 HTTP/1.1" 200 None 2026-03-11 01:56:34,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:56:34,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 01:56:34,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:56:34,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0ca159d19175818252c07 HTTP/1.1" 200 None 2026-03-11 01:56:35,090 - lava_callback - INFO - Completed processing callback for node 69b0ca159d19175818252c07 INFO: 172.21.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57406 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:56:57,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:56:57,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK 2026-03-11 01:58:13,370 - lava_callback - INFO - Received callback for node 69ae29328ef4424e487b4479 INFO: 172.21.0.1:52112 - "POST /node/69ae29328ef4424e487b4479 HTTP/1.0" 202 Accepted 2026-03-11 01:58:13,375 - lava_callback - INFO - Starting async job submit for node 69ae29328ef4424e487b4479 2026-03-11 01:58:13,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:58:13,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae29328ef4424e487b4479 HTTP/1.1" 200 None 2026-03-11 01:58:13,391 - lava_callback - DEBUG - Processing node 69ae29328ef4424e487b4479: baseline-arm64-baylibre 2026-03-11 01:58:13,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae29328ef4424e487b4479 2026-03-11 01:58:13,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:58:13,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae29328ef4424e487b4479 HTTP/1.1" 200 None 2026-03-11 01:58:13,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:58:13,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a628a67da105ad098359a5 HTTP/1.1" 200 None 2026-03-11 01:58:13,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 01:58:13,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae29328ef4424e487b4479 HTTP/1.1" 200 None 2026-03-11 01:58:13,445 - lava_callback - ERROR - Error processing callback for node 69ae29328ef4424e487b4479: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54510 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:04:18,688 - lava_callback - INFO - Received callback for node 69b0c96e9d19175818252beb INFO: 172.21.0.1:54516 - "POST /node/69b0c96e9d19175818252beb HTTP/1.0" 202 Accepted 2026-03-11 02:04:18,694 - lava_callback - INFO - Starting async job submit for node 69b0c96e9d19175818252beb 2026-03-11 02:04:18,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:04:18,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c96e9d19175818252beb HTTP/1.1" 200 None 2026-03-11 02:04:18,773 - lava_callback - DEBUG - Processing node 69b0c96e9d19175818252beb: ltp-smoke 2026-03-11 02:04:19,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:04:19,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:04:19,410 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0c96e9d19175818252beb/log.txt.gz 2026-03-11 02:04:19,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:04:19,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:04:19,911 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0c96e9d19175818252beb/lava-logs.yaml 2026-03-11 02:04:19,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:04:20,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:04:20,088 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0c96e9d19175818252beb/lava_callback.json.gz 2026-03-11 02:04:20,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:04:20,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0c96e9d19175818252beb HTTP/1.1" 200 None 2026-03-11 02:04:20,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:04:20,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5059d19175818251fdc HTTP/1.1" 200 None 2026-03-11 02:04:20,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:04:20,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0c96e9d19175818252beb HTTP/1.1" 200 None 2026-03-11 02:04:20,426 - lava_callback - INFO - Completed processing callback for node 69b0c96e9d19175818252beb INFO: 172.21.0.1:53244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53254 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:04:27,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:04:27,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36044 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:08:43,910 - lava_callback - INFO - Received callback for node 69b0cd759d19175818252c78 INFO: 172.21.0.1:54472 - "POST /node/69b0cd759d19175818252c78 HTTP/1.0" 202 Accepted 2026-03-11 02:08:43,917 - lava_callback - INFO - Starting async job submit for node 69b0cd759d19175818252c78 2026-03-11 02:08:43,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:08:43,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cd759d19175818252c78 HTTP/1.1" 200 None 2026-03-11 02:08:43,976 - lava_callback - DEBUG - Processing node 69b0cd759d19175818252c78: baseline-arm64 2026-03-11 02:08:44,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:08:44,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:08:44,518 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0cd759d19175818252c78/log.txt.gz 2026-03-11 02:08:44,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:08:44,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:08:44,944 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0cd759d19175818252c78/lava-logs.yaml 2026-03-11 02:08:44,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:08:45,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:08:45,153 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b0cd759d19175818252c78/lava_callback.json.gz DEBUG: Setup failed for 69b0cd759d19175818252c78. Transitting job node result: incomplete -> fail 2026-03-11 02:08:45,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:08:45,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cd759d19175818252c78 HTTP/1.1" 200 None 2026-03-11 02:08:45,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:08:45,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 02:08:45,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:08:45,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cd759d19175818252c78 HTTP/1.1" 200 None 2026-03-11 02:08:45,400 - lava_callback - INFO - Completed processing callback for node 69b0cd759d19175818252c78 INFO: 172.21.0.1:54478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46734 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:08:57,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:08:57,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:11:52,951 - lava_callback - INFO - Received callback for node 69b0cb4c9d19175818252c28 INFO: 172.21.0.1:54304 - "POST /node/69b0cb4c9d19175818252c28 HTTP/1.0" 202 Accepted 2026-03-11 02:11:52,958 - lava_callback - INFO - Starting async job submit for node 69b0cb4c9d19175818252c28 2026-03-11 02:11:53,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:11:53,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb4c9d19175818252c28 HTTP/1.1" 200 None 2026-03-11 02:11:53,034 - lava_callback - DEBUG - Processing node 69b0cb4c9d19175818252c28: baseline-arm 2026-03-11 02:11:53,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:11:53,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:11:53,779 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0cb4c9d19175818252c28/log.txt.gz 2026-03-11 02:11:54,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:11:54,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:11:54,387 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0cb4c9d19175818252c28/lava-logs.yaml 2026-03-11 02:11:54,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:11:54,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:11:54,576 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0cb4c9d19175818252c28/lava_callback.json.gz DEBUG: Setup failed for 69b0cb4c9d19175818252c28. Transitting job node result: incomplete -> fail 2026-03-11 02:11:54,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:11:54,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cb4c9d19175818252c28 HTTP/1.1" 200 None 2026-03-11 02:11:54,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:11:54,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 02:11:54,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:11:54,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cb4c9d19175818252c28 HTTP/1.1" 200 None 2026-03-11 02:11:54,915 - lava_callback - INFO - Completed processing callback for node 69b0cb4c9d19175818252c28 INFO: 172.21.0.1:54314 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:11:57,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:11:57,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34818 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:15:45,032 - lava_callback - INFO - Received callback for node 69b0cbc99d19175818252c3c INFO: 172.21.0.1:39658 - "POST /node/69b0cbc99d19175818252c3c HTTP/1.0" 202 Accepted 2026-03-11 02:15:45,083 - lava_callback - INFO - Starting async job submit for node 69b0cbc99d19175818252c3c 2026-03-11 02:15:45,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:15:45,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cbc99d19175818252c3c HTTP/1.1" 200 None 2026-03-11 02:15:45,130 - lava_callback - DEBUG - Processing node 69b0cbc99d19175818252c3c: baseline-arm-pengutronix 2026-03-11 02:15:45,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:15:45,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:15:45,400 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0cbc99d19175818252c3c/log.txt.gz 2026-03-11 02:15:45,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:15:45,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:15:45,647 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0cbc99d19175818252c3c/lava-logs.yaml 2026-03-11 02:15:45,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:15:45,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:15:45,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0cbc99d19175818252c3c/lava_callback.json.gz DEBUG: Setup failed for 69b0cbc99d19175818252c3c. Transitting job node result: incomplete -> fail 2026-03-11 02:15:45,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:15:45,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cbc99d19175818252c3c HTTP/1.1" 200 None 2026-03-11 02:15:45,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:15:45,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 02:15:45,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:15:45,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cbc99d19175818252c3c HTTP/1.1" 200 None 2026-03-11 02:15:46,001 - lava_callback - INFO - Completed processing callback for node 69b0cbc99d19175818252c3c INFO: 172.21.0.1:39674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40470 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:15:57,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:15:57,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39858 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:21:58,713 - lava_callback - INFO - Received callback for node 69b0cd289d19175818252c6f INFO: 172.21.0.1:39866 - "POST /node/69b0cd289d19175818252c6f HTTP/1.0" 202 Accepted 2026-03-11 02:21:58,721 - lava_callback - INFO - Starting async job submit for node 69b0cd289d19175818252c6f 2026-03-11 02:21:58,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:21:58,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cd289d19175818252c6f HTTP/1.1" 200 None 2026-03-11 02:21:58,794 - lava_callback - DEBUG - Processing node 69b0cd289d19175818252c6f: baseline-nfs-arm64 2026-03-11 02:21:59,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:22:00,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:22:00,110 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0cd289d19175818252c6f/log.txt.gz 2026-03-11 02:22:00,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:22:01,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:22:01,028 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0cd289d19175818252c6f/lava-logs.yaml 2026-03-11 02:22:01,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:22:01,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:22:01,250 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0cd289d19175818252c6f/lava_callback.json.gz DEBUG: Setup failed for 69b0cd289d19175818252c6f. Transitting job node result: incomplete -> fail 2026-03-11 02:22:01,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:22:01,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cd289d19175818252c6f HTTP/1.1" 200 None 2026-03-11 02:22:01,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:22:01,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 02:22:01,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:22:01,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cd289d19175818252c6f HTTP/1.1" 200 None 2026-03-11 02:22:01,522 - lava_callback - INFO - Completed processing callback for node 69b0cd289d19175818252c6f INFO: 172.21.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35850 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:22:27,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:22:28,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:22:59,078 - lava_callback - INFO - Received callback for node 69b0cdbf9d19175818252c86 INFO: 172.21.0.1:60370 - "POST /node/69b0cdbf9d19175818252c86 HTTP/1.0" 202 Accepted 2026-03-11 02:22:59,084 - lava_callback - INFO - Starting async job submit for node 69b0cdbf9d19175818252c86 2026-03-11 02:22:59,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:22:59,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cdbf9d19175818252c86 HTTP/1.1" 200 None 2026-03-11 02:22:59,157 - lava_callback - DEBUG - Processing node 69b0cdbf9d19175818252c86: ltp-smoke 2026-03-11 02:22:59,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:22:59,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:22:59,758 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0cdbf9d19175818252c86/log.txt.gz 2026-03-11 02:23:00,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:23:00,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:23:00,268 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0cdbf9d19175818252c86/lava-logs.yaml 2026-03-11 02:23:00,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:23:00,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:23:00,448 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b0cdbf9d19175818252c86/lava_callback.json.gz 2026-03-11 02:23:00,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:23:00,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cdbf9d19175818252c86 HTTP/1.1" 200 None 2026-03-11 02:23:00,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:23:00,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5059d19175818251fdc HTTP/1.1" 200 None 2026-03-11 02:23:00,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:23:00,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cdbf9d19175818252c86 HTTP/1.1" 200 None 2026-03-11 02:23:00,753 - lava_callback - INFO - Completed processing callback for node 69b0cdbf9d19175818252c86 INFO: 172.21.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:23:28,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:23:28,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57168 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:29:31,223 - lava_callback - INFO - Received callback for node 69b0cf6c9d19175818252c95 INFO: 172.21.0.1:40996 - "POST /node/69b0cf6c9d19175818252c95 HTTP/1.0" 202 Accepted 2026-03-11 02:29:31,230 - lava_callback - INFO - Starting async job submit for node 69b0cf6c9d19175818252c95 2026-03-11 02:29:31,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:29:31,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cf6c9d19175818252c95 HTTP/1.1" 200 None 2026-03-11 02:29:31,309 - lava_callback - DEBUG - Processing node 69b0cf6c9d19175818252c95: baseline-arm 2026-03-11 02:29:31,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:29:32,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:29:32,058 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b0cf6c9d19175818252c95/log.txt.gz 2026-03-11 02:29:32,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:29:32,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:29:32,645 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b0cf6c9d19175818252c95/lava-logs.yaml 2026-03-11 02:29:32,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:29:32,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:29:32,832 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b0cf6c9d19175818252c95/lava_callback.json.gz DEBUG: Setup failed for 69b0cf6c9d19175818252c95. Transitting job node result: incomplete -> fail 2026-03-11 02:29:32,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:29:32,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0cf6c9d19175818252c95 HTTP/1.1" 200 None 2026-03-11 02:29:32,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:29:32,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 02:29:32,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:29:33,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0cf6c9d19175818252c95 HTTP/1.1" 200 None 2026-03-11 02:29:33,130 - lava_callback - INFO - Completed processing callback for node 69b0cf6c9d19175818252c95 INFO: 172.21.0.1:41012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47422 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:29:58,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:29:58,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50462 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:41:57,635 - lava_callback - INFO - Received callback for node 69b0d1d29d19175818252cb6 INFO: 172.21.0.1:50464 - "POST /node/69b0d1d29d19175818252cb6 HTTP/1.0" 202 Accepted 2026-03-11 02:41:57,643 - lava_callback - INFO - Starting async job submit for node 69b0d1d29d19175818252cb6 2026-03-11 02:41:57,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:41:57,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d1d29d19175818252cb6 HTTP/1.1" 200 None 2026-03-11 02:41:57,709 - lava_callback - DEBUG - Processing node 69b0d1d29d19175818252cb6: baseline-nfs-arm64 2026-03-11 02:41:58,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:41:59,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:41:59,007 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0d1d29d19175818252cb6/log.txt.gz 2026-03-11 02:41:59,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:41:59,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:41:59,944 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0d1d29d19175818252cb6/lava-logs.yaml 2026-03-11 02:41:59,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:42:00,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:42:00,166 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b0d1d29d19175818252cb6/lava_callback.json.gz DEBUG: Setup failed for 69b0d1d29d19175818252cb6. Transitting job node result: incomplete -> fail 2026-03-11 02:42:00,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:42:00,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d1d29d19175818252cb6 HTTP/1.1" 200 None 2026-03-11 02:42:00,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:42:00,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 02:42:00,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:42:00,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d1d29d19175818252cb6 HTTP/1.1" 200 None 2026-03-11 02:42:00,417 - lava_callback - INFO - Completed processing callback for node 69b0d1d29d19175818252cb6 INFO: 172.21.0.1:45240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44092 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:42:28,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:42:28,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36734 - "GET /node/698f5cda3013abef95cf8c1a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:36738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:49:56,200 - lava_callback - INFO - Received callback for node 69ae35478ef4424e487b47b9 INFO: 172.21.0.1:42684 - "POST /node/69ae35478ef4424e487b47b9 HTTP/1.0" 202 Accepted 2026-03-11 02:49:56,206 - lava_callback - INFO - Starting async job submit for node 69ae35478ef4424e487b47b9 2026-03-11 02:49:56,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:49:56,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae35478ef4424e487b47b9 HTTP/1.1" 200 None 2026-03-11 02:49:56,222 - lava_callback - DEBUG - Processing node 69ae35478ef4424e487b47b9: baseline-arm64-baylibre 2026-03-11 02:49:56,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae35478ef4424e487b47b9 2026-03-11 02:49:56,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:49:56,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae35478ef4424e487b47b9 HTTP/1.1" 200 None 2026-03-11 02:49:56,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:49:56,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 02:49:56,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:49:56,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae35478ef4424e487b47b9 HTTP/1.1" 200 None 2026-03-11 02:49:56,305 - lava_callback - ERROR - Error processing callback for node 69ae35478ef4424e487b47b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51656 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:51:56,246 - lava_callback - INFO - Received callback for node 69ae35c08ef4424e487b47d4 INFO: 172.21.0.1:45504 - "POST /node/69ae35c08ef4424e487b47d4 HTTP/1.0" 202 Accepted 2026-03-11 02:51:56,251 - lava_callback - INFO - Starting async job submit for node 69ae35c08ef4424e487b47d4 2026-03-11 02:51:56,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:51:56,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae35c08ef4424e487b47d4 HTTP/1.1" 200 None 2026-03-11 02:51:56,277 - lava_callback - DEBUG - Processing node 69ae35c08ef4424e487b47d4: baseline-arm64-baylibre 2026-03-11 02:51:56,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae35c08ef4424e487b47d4 2026-03-11 02:51:56,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:51:56,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae35c08ef4424e487b47d4 HTTP/1.1" 200 None 2026-03-11 02:51:56,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:51:56,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 02:51:56,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:51:56,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae35c08ef4424e487b47d4 HTTP/1.1" 200 None 2026-03-11 02:51:56,330 - lava_callback - ERROR - Error processing callback for node 69ae35c08ef4424e487b47d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-11 02:53:16,414 - lava_callback - INFO - Received callback for node 69ae36138ef4424e487b47e6 INFO: 172.21.0.1:40240 - "POST /node/69ae36138ef4424e487b47e6 HTTP/1.0" 202 Accepted 2026-03-11 02:53:16,419 - lava_callback - INFO - Starting async job submit for node 69ae36138ef4424e487b47e6 2026-03-11 02:53:16,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:53:16,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae36138ef4424e487b47e6 HTTP/1.1" 200 None 2026-03-11 02:53:16,439 - lava_callback - DEBUG - Processing node 69ae36138ef4424e487b47e6: baseline-arm64-baylibre 2026-03-11 02:53:16,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae36138ef4424e487b47e6 2026-03-11 02:53:16,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:53:16,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae36138ef4424e487b47e6 HTTP/1.1" 200 None 2026-03-11 02:53:16,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:53:16,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 02:53:16,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:53:16,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae36138ef4424e487b47e6 HTTP/1.1" 200 None 2026-03-11 02:53:16,490 - lava_callback - ERROR - Error processing callback for node 69ae36138ef4424e487b47e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34856 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:54:26,398 - lava_callback - INFO - Received callback for node 69b0d05f9d19175818252cad INFO: 172.21.0.1:58620 - "POST /node/69b0d05f9d19175818252cad HTTP/1.0" 202 Accepted 2026-03-11 02:54:26,446 - lava_callback - INFO - Starting async job submit for node 69b0d05f9d19175818252cad 2026-03-11 02:54:26,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:54:26,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d05f9d19175818252cad HTTP/1.1" 200 None 2026-03-11 02:54:26,491 - lava_callback - DEBUG - Processing node 69b0d05f9d19175818252cad: baseline-arm-pengutronix INFO: 172.21.0.1:58632 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:54:26,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:54:26,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:54:26,749 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0d05f9d19175818252cad/log.txt.gz 2026-03-11 02:54:26,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:58642 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:54:26,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:54:26,985 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0d05f9d19175818252cad/lava-logs.yaml 2026-03-11 02:54:26,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 02:54:27,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 02:54:27,155 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b0d05f9d19175818252cad/lava_callback.json.gz DEBUG: Setup failed for 69b0d05f9d19175818252cad. Transitting job node result: incomplete -> fail 2026-03-11 02:54:27,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:54:27,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0d05f9d19175818252cad HTTP/1.1" 200 None 2026-03-11 02:54:27,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:54:27,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b4fc9d19175818251fd6 HTTP/1.1" 200 None 2026-03-11 02:54:27,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:54:27,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0d05f9d19175818252cad HTTP/1.1" 200 None 2026-03-11 02:54:27,320 - lava_callback - INFO - Completed processing callback for node 69b0d05f9d19175818252cad 2026-03-11 02:54:28,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:54:28,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:56:06,438 - lava_callback - INFO - Received callback for node 69ae36bd8ef4424e487b4817 INFO: 172.21.0.1:42188 - "POST /node/69ae36bd8ef4424e487b4817 HTTP/1.0" 202 Accepted 2026-03-11 02:56:06,444 - lava_callback - INFO - Starting async job submit for node 69ae36bd8ef4424e487b4817 2026-03-11 02:56:06,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:56:06,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae36bd8ef4424e487b4817 HTTP/1.1" 200 None 2026-03-11 02:56:06,459 - lava_callback - DEBUG - Processing node 69ae36bd8ef4424e487b4817: baseline-arm64-clabbe 2026-03-11 02:56:06,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae36bd8ef4424e487b4817 2026-03-11 02:56:06,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:56:06,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae36bd8ef4424e487b4817 HTTP/1.1" 200 None 2026-03-11 02:56:06,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:56:06,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 02:56:06,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:42196 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:56:06,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae36bd8ef4424e487b4817 HTTP/1.1" 200 None 2026-03-11 02:56:06,517 - lava_callback - ERROR - Error processing callback for node 69ae36bd8ef4424e487b4817: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59174 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:57:26,507 - lava_callback - INFO - Received callback for node 69ae370f8ef4424e487b483c INFO: 172.21.0.1:34836 - "POST /node/69ae370f8ef4424e487b483c HTTP/1.0" 202 Accepted 2026-03-11 02:57:26,512 - lava_callback - INFO - Starting async job submit for node 69ae370f8ef4424e487b483c INFO: 172.21.0.1:34842 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:57:26,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:57:26,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae370f8ef4424e487b483c HTTP/1.1" 200 None 2026-03-11 02:57:26,535 - lava_callback - DEBUG - Processing node 69ae370f8ef4424e487b483c: baseline-arm64-clabbe 2026-03-11 02:57:26,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae370f8ef4424e487b483c 2026-03-11 02:57:26,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:57:26,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae370f8ef4424e487b483c HTTP/1.1" 200 None 2026-03-11 02:57:26,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:57:26,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 02:57:26,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:57:26,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae370f8ef4424e487b483c HTTP/1.1" 200 None 2026-03-11 02:57:26,588 - lava_callback - ERROR - Error processing callback for node 69ae370f8ef4424e487b483c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47424 - "GET / HTTP/1.0" 200 OK 2026-03-11 02:58:56,605 - lava_callback - INFO - Received callback for node 69ae375b8ef4424e487b4848 INFO: 172.21.0.1:47436 - "POST /node/69ae375b8ef4424e487b4848 HTTP/1.0" 202 Accepted 2026-03-11 02:58:56,610 - lava_callback - INFO - Starting async job submit for node 69ae375b8ef4424e487b4848 2026-03-11 02:58:56,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:58:56,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae375b8ef4424e487b4848 HTTP/1.1" 200 None 2026-03-11 02:58:56,648 - lava_callback - DEBUG - Processing node 69ae375b8ef4424e487b4848: baseline-arm64-baylibre 2026-03-11 02:58:56,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae375b8ef4424e487b4848 2026-03-11 02:58:56,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:58:56,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae375b8ef4424e487b4848 HTTP/1.1" 200 None 2026-03-11 02:58:56,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:58:56,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 02:58:56,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 02:58:56,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae375b8ef4424e487b4848 HTTP/1.1" 200 None 2026-03-11 02:58:56,717 - lava_callback - ERROR - Error processing callback for node 69ae375b8ef4424e487b4848: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60680 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:00:46,644 - lava_callback - INFO - Received callback for node 69ae37c98ef4424e487b486c INFO: 172.21.0.1:60696 - "POST /node/69ae37c98ef4424e487b486c HTTP/1.0" 202 Accepted 2026-03-11 03:00:46,650 - lava_callback - INFO - Starting async job submit for node 69ae37c98ef4424e487b486c 2026-03-11 03:00:46,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:00:46,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae37c98ef4424e487b486c HTTP/1.1" 200 None 2026-03-11 03:00:46,667 - lava_callback - DEBUG - Processing node 69ae37c98ef4424e487b486c: baseline-arm64-clabbe 2026-03-11 03:00:46,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae37c98ef4424e487b486c 2026-03-11 03:00:46,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:00:46,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae37c98ef4424e487b486c HTTP/1.1" 200 None 2026-03-11 03:00:46,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:00:46,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 03:00:46,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:00:46,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae37c98ef4424e487b486c HTTP/1.1" 200 None 2026-03-11 03:00:46,719 - lava_callback - ERROR - Error processing callback for node 69ae37c98ef4424e487b486c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37680 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:02:06,729 - lava_callback - INFO - Received callback for node 69ae381c8ef4424e487b4889 INFO: 172.21.0.1:37692 - "POST /node/69ae381c8ef4424e487b4889 HTTP/1.0" 202 Accepted 2026-03-11 03:02:06,735 - lava_callback - INFO - Starting async job submit for node 69ae381c8ef4424e487b4889 2026-03-11 03:02:06,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:02:06,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae381c8ef4424e487b4889 HTTP/1.1" 200 None 2026-03-11 03:02:06,753 - lava_callback - DEBUG - Processing node 69ae381c8ef4424e487b4889: baseline-arm64-clabbe 2026-03-11 03:02:06,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae381c8ef4424e487b4889 2026-03-11 03:02:06,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:02:06,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae381c8ef4424e487b4889 HTTP/1.1" 200 None 2026-03-11 03:02:06,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:02:06,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafea227cb11dcdfc758f1 HTTP/1.1" 200 None 2026-03-11 03:02:06,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:02:06,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae381c8ef4424e487b4889 HTTP/1.1" 200 None 2026-03-11 03:02:06,833 - lava_callback - ERROR - Error processing callback for node 69ae381c8ef4424e487b4889: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:04:16,916 - lava_callback - INFO - Received callback for node 69ae38af8ef4424e487b4903 INFO: 172.21.0.1:48578 - "POST /node/69ae38af8ef4424e487b4903 HTTP/1.0" 202 Accepted 2026-03-11 03:04:16,922 - lava_callback - INFO - Starting async job submit for node 69ae38af8ef4424e487b4903 2026-03-11 03:04:16,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:04:16,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae38af8ef4424e487b4903 HTTP/1.1" 200 None 2026-03-11 03:04:16,938 - lava_callback - DEBUG - Processing node 69ae38af8ef4424e487b4903: baseline-arm-baylibre 2026-03-11 03:04:16,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae38af8ef4424e487b4903 2026-03-11 03:04:16,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:04:16,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae38af8ef4424e487b4903 HTTP/1.1" 200 None 2026-03-11 03:04:16,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:04:16,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-11 03:04:16,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:04:16,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae38af8ef4424e487b4903 HTTP/1.1" 200 None 2026-03-11 03:04:16,994 - lava_callback - ERROR - Error processing callback for node 69ae38af8ef4424e487b4903: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57526 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:06:26,878 - lava_callback - INFO - Received callback for node 69ae392a8ef4424e487b49a6 INFO: 172.21.0.1:57530 - "POST /node/69ae392a8ef4424e487b49a6 HTTP/1.0" 202 Accepted 2026-03-11 03:06:26,884 - lava_callback - INFO - Starting async job submit for node 69ae392a8ef4424e487b49a6 2026-03-11 03:06:26,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:06:26,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae392a8ef4424e487b49a6 HTTP/1.1" 200 None 2026-03-11 03:06:26,912 - lava_callback - DEBUG - Processing node 69ae392a8ef4424e487b49a6: baseline-arm-clabbe 2026-03-11 03:06:26,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae392a8ef4424e487b49a6 2026-03-11 03:06:26,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:06:26,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae392a8ef4424e487b49a6 HTTP/1.1" 200 None 2026-03-11 03:06:26,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:06:26,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-11 03:06:26,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:06:26,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae392a8ef4424e487b49a6 HTTP/1.1" 200 None 2026-03-11 03:06:26,970 - lava_callback - ERROR - Error processing callback for node 69ae392a8ef4424e487b49a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56556 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:07:06,916 - lava_callback - INFO - Received callback for node 69ae39548ef4424e487b49d3 INFO: 172.21.0.1:56558 - "POST /node/69ae39548ef4424e487b49d3 HTTP/1.0" 202 Accepted 2026-03-11 03:07:06,921 - lava_callback - INFO - Starting async job submit for node 69ae39548ef4424e487b49d3 2026-03-11 03:07:06,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:06,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae39548ef4424e487b49d3 HTTP/1.1" 200 None 2026-03-11 03:07:06,937 - lava_callback - DEBUG - Processing node 69ae39548ef4424e487b49d3: baseline-arm-clabbe 2026-03-11 03:07:06,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae39548ef4424e487b49d3 2026-03-11 03:07:06,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:06,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae39548ef4424e487b49d3 HTTP/1.1" 200 None 2026-03-11 03:07:06,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:06,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-11 03:07:06,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:07,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae39548ef4424e487b49d3 HTTP/1.1" 200 None 2026-03-11 03:07:07,017 - lava_callback - ERROR - Error processing callback for node 69ae39548ef4424e487b49d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47428 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:07:46,939 - lava_callback - INFO - Received callback for node 69ae397d8ef4424e487b4a20 INFO: 172.21.0.1:47444 - "POST /node/69ae397d8ef4424e487b4a20 HTTP/1.0" 202 Accepted 2026-03-11 03:07:46,944 - lava_callback - INFO - Starting async job submit for node 69ae397d8ef4424e487b4a20 2026-03-11 03:07:46,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:46,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae397d8ef4424e487b4a20 HTTP/1.1" 200 None 2026-03-11 03:07:46,960 - lava_callback - DEBUG - Processing node 69ae397d8ef4424e487b4a20: baseline-arm-clabbe 2026-03-11 03:07:46,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae397d8ef4424e487b4a20 2026-03-11 03:07:46,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:46,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae397d8ef4424e487b4a20 HTTP/1.1" 200 None 2026-03-11 03:07:46,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:47,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-11 03:07:47,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:07:47,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae397d8ef4424e487b4a20 HTTP/1.1" 200 None 2026-03-11 03:07:47,025 - lava_callback - ERROR - Error processing callback for node 69ae397d8ef4424e487b4a20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47098 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:28:18,304 - lava_callback - INFO - Received callback for node 69ae3e4a8ef4424e487b4b95 INFO: 172.21.0.1:47100 - "POST /node/69ae3e4a8ef4424e487b4b95 HTTP/1.0" 202 Accepted 2026-03-11 03:28:18,310 - lava_callback - INFO - Starting async job submit for node 69ae3e4a8ef4424e487b4b95 2026-03-11 03:28:18,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:28:18,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3e4a8ef4424e487b4b95 HTTP/1.1" 200 None 2026-03-11 03:28:18,331 - lava_callback - DEBUG - Processing node 69ae3e4a8ef4424e487b4b95: baseline-arm64-baylibre 2026-03-11 03:28:18,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae3e4a8ef4424e487b4b95 2026-03-11 03:28:18,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:28:18,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3e4a8ef4424e487b4b95 HTTP/1.1" 200 None 2026-03-11 03:28:18,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:28:18,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-11 03:28:18,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:28:18,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae3e4a8ef4424e487b4b95 HTTP/1.1" 200 None 2026-03-11 03:28:18,389 - lava_callback - ERROR - Error processing callback for node 69ae3e4a8ef4424e487b4b95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46322 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:32:58,449 - lava_callback - INFO - Received callback for node 69ae3f608ef4424e487b4be6 INFO: 172.21.0.1:46336 - "POST /node/69ae3f608ef4424e487b4be6 HTTP/1.0" 202 Accepted 2026-03-11 03:32:58,455 - lava_callback - INFO - Starting async job submit for node 69ae3f608ef4424e487b4be6 2026-03-11 03:32:58,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:32:58,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3f608ef4424e487b4be6 HTTP/1.1" 200 None 2026-03-11 03:32:58,478 - lava_callback - DEBUG - Processing node 69ae3f608ef4424e487b4be6: baseline-arm64-baylibre 2026-03-11 03:32:58,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae3f608ef4424e487b4be6 2026-03-11 03:32:58,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:32:58,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3f608ef4424e487b4be6 HTTP/1.1" 200 None 2026-03-11 03:32:58,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:32:58,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-11 03:32:58,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:32:58,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae3f608ef4424e487b4be6 HTTP/1.1" 200 None 2026-03-11 03:32:58,534 - lava_callback - ERROR - Error processing callback for node 69ae3f608ef4424e487b4be6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39130 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:34:28,071 - lava_callback - INFO - Received callback for node 69ae3fb98ef4424e487b4c00 INFO: 172.21.0.1:39144 - "POST /node/69ae3fb98ef4424e487b4c00 HTTP/1.0" 202 Accepted 2026-03-11 03:34:28,076 - lava_callback - INFO - Starting async job submit for node 69ae3fb98ef4424e487b4c00 2026-03-11 03:34:28,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:34:28,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3fb98ef4424e487b4c00 HTTP/1.1" 200 None 2026-03-11 03:34:28,092 - lava_callback - DEBUG - Processing node 69ae3fb98ef4424e487b4c00: baseline-arm64-clabbe 2026-03-11 03:34:28,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae3fb98ef4424e487b4c00 2026-03-11 03:34:28,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:34:28,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae3fb98ef4424e487b4c00 HTTP/1.1" 200 None 2026-03-11 03:34:28,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:34:28,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-11 03:34:28,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:34:28,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae3fb98ef4424e487b4c00 HTTP/1.1" 200 None 2026-03-11 03:34:28,180 - lava_callback - ERROR - Error processing callback for node 69ae3fb98ef4424e487b4c00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58626 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:35:48,051 - lava_callback - INFO - Received callback for node 69ae400d8ef4424e487b4c1a INFO: 172.21.0.1:58636 - "POST /node/69ae400d8ef4424e487b4c1a HTTP/1.0" 202 Accepted 2026-03-11 03:35:48,057 - lava_callback - INFO - Starting async job submit for node 69ae400d8ef4424e487b4c1a 2026-03-11 03:35:48,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:35:48,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae400d8ef4424e487b4c1a HTTP/1.1" 200 None 2026-03-11 03:35:48,074 - lava_callback - DEBUG - Processing node 69ae400d8ef4424e487b4c1a: baseline-arm64-clabbe 2026-03-11 03:35:48,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae400d8ef4424e487b4c1a 2026-03-11 03:35:48,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:35:48,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae400d8ef4424e487b4c1a HTTP/1.1" 200 None 2026-03-11 03:35:48,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:35:48,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-11 03:35:48,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:35:48,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae400d8ef4424e487b4c1a HTTP/1.1" 200 None 2026-03-11 03:35:48,132 - lava_callback - ERROR - Error processing callback for node 69ae400d8ef4424e487b4c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57196 - "GET /node/698f5b4f3013abef95cf7c28%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50112 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:41:42,042 - lava_callback - INFO - Received callback for node 69b00955ae557f2600f10700 INFO: 172.21.0.1:46762 - "POST /node/69b00955ae557f2600f10700 HTTP/1.0" 202 Accepted 2026-03-11 03:41:42,048 - lava_callback - INFO - Starting async job submit for node 69b00955ae557f2600f10700 2026-03-11 03:41:42,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:41:42,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00955ae557f2600f10700 HTTP/1.1" 200 None 2026-03-11 03:41:42,109 - lava_callback - DEBUG - Processing node 69b00955ae557f2600f10700: baseline-arm64-broonie 2026-03-11 03:41:42,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:41:42,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:41:42,637 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b00955ae557f2600f10700/log.txt.gz 2026-03-11 03:41:42,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:41:43,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:41:43,082 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b00955ae557f2600f10700/lava-logs.yaml 2026-03-11 03:41:43,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:41:43,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:41:43,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b00955ae557f2600f10700/lava_callback.json.gz 2026-03-11 03:41:43,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:41:43,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00955ae557f2600f10700 HTTP/1.1" 200 None 2026-03-11 03:41:43,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:41:43,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-11 03:41:43,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:41:43,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00955ae557f2600f10700 HTTP/1.1" 200 None 2026-03-11 03:41:43,305 - lava_callback - INFO - Completed processing callback for node 69b00955ae557f2600f10700 INFO: 172.21.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:41:58,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:41:58,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:44:42,092 - lava_callback - INFO - Received callback for node 69b0be379d19175818252829 INFO: 172.21.0.1:54156 - "POST /node/69b0be379d19175818252829 HTTP/1.0" 202 Accepted 2026-03-11 03:44:42,098 - lava_callback - INFO - Starting async job submit for node 69b0be379d19175818252829 2026-03-11 03:44:42,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:44:42,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be379d19175818252829 HTTP/1.1" 200 None 2026-03-11 03:44:42,132 - lava_callback - DEBUG - Processing node 69b0be379d19175818252829: baseline-arm64-broonie 2026-03-11 03:44:42,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:44:42,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:44:42,625 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be379d19175818252829/log.txt.gz 2026-03-11 03:44:42,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:44:43,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:44:43,035 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be379d19175818252829/lava-logs.yaml 2026-03-11 03:44:43,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:44:43,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:44:43,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be379d19175818252829/lava_callback.json.gz 2026-03-11 03:44:43,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:44:43,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be379d19175818252829 HTTP/1.1" 200 None 2026-03-11 03:44:43,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:44:43,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 03:44:43,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:44:43,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be379d19175818252829 HTTP/1.1" 200 None 2026-03-11 03:44:43,350 - lava_callback - INFO - Completed processing callback for node 69b0be379d19175818252829 INFO: 172.21.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37556 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:44:58,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:44:58,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45768 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:46:42,159 - lava_callback - INFO - Received callback for node 69b0be3d9d19175818252836 INFO: 172.21.0.1:43294 - "POST /node/69b0be3d9d19175818252836 HTTP/1.0" 202 Accepted 2026-03-11 03:46:42,165 - lava_callback - INFO - Starting async job submit for node 69b0be3d9d19175818252836 2026-03-11 03:46:42,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:46:42,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be3d9d19175818252836 HTTP/1.1" 200 None 2026-03-11 03:46:42,212 - lava_callback - DEBUG - Processing node 69b0be3d9d19175818252836: baseline-arm64-broonie 2026-03-11 03:46:42,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:46:43,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:46:43,572 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3d9d19175818252836/log.txt.gz 2026-03-11 03:46:43,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:46:43,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:46:43,939 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3d9d19175818252836/lava-logs.yaml 2026-03-11 03:46:43,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-11 03:46:44,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-11 03:46:44,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b0be3d9d19175818252836/lava_callback.json.gz 2026-03-11 03:46:44,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:46:44,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0be3d9d19175818252836 HTTP/1.1" 200 None 2026-03-11 03:46:44,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:46:44,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0b5149d19175818251feb HTTP/1.1" 200 None 2026-03-11 03:46:44,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:46:44,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0be3d9d19175818252836 HTTP/1.1" 200 None 2026-03-11 03:46:44,165 - lava_callback - INFO - Completed processing callback for node 69b0be3d9d19175818252836 INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33254 - "GET / HTTP/1.0" 200 OK 2026-03-11 03:46:58,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 03:46:58,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43150 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58664 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39378 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:47:22,617 - lava_callback - INFO - Received callback for node 69ae50c68ef4424e487b4f7e INFO: 172.21.0.1:51704 - "POST /node/69ae50c68ef4424e487b4f7e HTTP/1.0" 202 Accepted 2026-03-11 04:47:22,623 - lava_callback - INFO - Starting async job submit for node 69ae50c68ef4424e487b4f7e 2026-03-11 04:47:22,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:47:22,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae50c68ef4424e487b4f7e HTTP/1.1" 200 None 2026-03-11 04:47:22,642 - lava_callback - DEBUG - Processing node 69ae50c68ef4424e487b4f7e: baseline-arm-baylibre 2026-03-11 04:47:22,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae50c68ef4424e487b4f7e 2026-03-11 04:47:22,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:47:22,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae50c68ef4424e487b4f7e HTTP/1.1" 200 None 2026-03-11 04:47:22,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:47:22,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:47:22,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:47:22,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae50c68ef4424e487b4f7e HTTP/1.1" 200 None 2026-03-11 04:47:22,696 - lava_callback - ERROR - Error processing callback for node 69ae50c68ef4424e487b4f7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43582 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:48:02,482 - lava_callback - INFO - Received callback for node 69ae50ef8ef4424e487b4fce INFO: 172.21.0.1:38976 - "POST /node/69ae50ef8ef4424e487b4fce HTTP/1.0" 202 Accepted 2026-03-11 04:48:02,487 - lava_callback - INFO - Starting async job submit for node 69ae50ef8ef4424e487b4fce 2026-03-11 04:48:02,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:02,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae50ef8ef4424e487b4fce HTTP/1.1" 200 None 2026-03-11 04:48:02,504 - lava_callback - DEBUG - Processing node 69ae50ef8ef4424e487b4fce: baseline-arm-baylibre 2026-03-11 04:48:02,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae50ef8ef4424e487b4fce 2026-03-11 04:48:02,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:02,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae50ef8ef4424e487b4fce HTTP/1.1" 200 None 2026-03-11 04:48:02,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:02,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:48:02,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:02,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae50ef8ef4424e487b4fce HTTP/1.1" 200 None 2026-03-11 04:48:02,556 - lava_callback - ERROR - Error processing callback for node 69ae50ef8ef4424e487b4fce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44458 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:48:42,573 - lava_callback - INFO - Received callback for node 69ae51198ef4424e487b4ffb INFO: 172.21.0.1:38100 - "POST /node/69ae51198ef4424e487b4ffb HTTP/1.0" 202 Accepted 2026-03-11 04:48:42,579 - lava_callback - INFO - Starting async job submit for node 69ae51198ef4424e487b4ffb 2026-03-11 04:48:42,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:42,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae51198ef4424e487b4ffb HTTP/1.1" 200 None 2026-03-11 04:48:42,595 - lava_callback - DEBUG - Processing node 69ae51198ef4424e487b4ffb: baseline-arm-baylibre 2026-03-11 04:48:42,595 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae51198ef4424e487b4ffb 2026-03-11 04:48:42,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:42,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae51198ef4424e487b4ffb HTTP/1.1" 200 None 2026-03-11 04:48:42,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:42,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:48:42,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:48:42,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae51198ef4424e487b4ffb HTTP/1.1" 200 None 2026-03-11 04:48:42,653 - lava_callback - ERROR - Error processing callback for node 69ae51198ef4424e487b4ffb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:49:22,545 - lava_callback - INFO - Received callback for node 69ae51438ef4424e487b5028 INFO: 172.21.0.1:46620 - "POST /node/69ae51438ef4424e487b5028 HTTP/1.0" 202 Accepted 2026-03-11 04:49:22,550 - lava_callback - INFO - Starting async job submit for node 69ae51438ef4424e487b5028 2026-03-11 04:49:22,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:49:22,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae51438ef4424e487b5028 HTTP/1.1" 200 None 2026-03-11 04:49:22,571 - lava_callback - DEBUG - Processing node 69ae51438ef4424e487b5028: baseline-arm-baylibre 2026-03-11 04:49:22,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae51438ef4424e487b5028 2026-03-11 04:49:22,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:49:22,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae51438ef4424e487b5028 HTTP/1.1" 200 None 2026-03-11 04:49:22,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:49:22,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:49:22,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:49:22,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae51438ef4424e487b5028 HTTP/1.1" 200 None 2026-03-11 04:49:22,622 - lava_callback - ERROR - Error processing callback for node 69ae51438ef4424e487b5028: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54440 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:50:02,664 - lava_callback - INFO - Received callback for node 69ae516d8ef4424e487b5079 INFO: 172.21.0.1:40492 - "POST /node/69ae516d8ef4424e487b5079 HTTP/1.0" 202 Accepted 2026-03-11 04:50:02,669 - lava_callback - INFO - Starting async job submit for node 69ae516d8ef4424e487b5079 2026-03-11 04:50:02,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:02,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae516d8ef4424e487b5079 HTTP/1.1" 200 None 2026-03-11 04:50:02,685 - lava_callback - DEBUG - Processing node 69ae516d8ef4424e487b5079: baseline-arm-baylibre 2026-03-11 04:50:02,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae516d8ef4424e487b5079 2026-03-11 04:50:02,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:02,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae516d8ef4424e487b5079 HTTP/1.1" 200 None 2026-03-11 04:50:02,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:02,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:50:02,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:02,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae516d8ef4424e487b5079 HTTP/1.1" 200 None 2026-03-11 04:50:02,739 - lava_callback - ERROR - Error processing callback for node 69ae516d8ef4424e487b5079: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57382 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:50:42,613 - lava_callback - INFO - Received callback for node 69ae51968ef4424e487b5096 INFO: 172.21.0.1:47838 - "POST /node/69ae51968ef4424e487b5096 HTTP/1.0" 202 Accepted 2026-03-11 04:50:42,619 - lava_callback - INFO - Starting async job submit for node 69ae51968ef4424e487b5096 2026-03-11 04:50:42,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:42,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae51968ef4424e487b5096 HTTP/1.1" 200 None 2026-03-11 04:50:42,635 - lava_callback - DEBUG - Processing node 69ae51968ef4424e487b5096: baseline-arm-baylibre 2026-03-11 04:50:42,635 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae51968ef4424e487b5096 2026-03-11 04:50:42,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:42,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae51968ef4424e487b5096 HTTP/1.1" 200 None 2026-03-11 04:50:42,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:42,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:50:42,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:50:42,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae51968ef4424e487b5096 HTTP/1.1" 200 None 2026-03-11 04:50:42,698 - lava_callback - ERROR - Error processing callback for node 69ae51968ef4424e487b5096: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:52:51,604 - lava_callback - INFO - Received callback for node 69ae52158ef4424e487b5134 INFO: 172.21.0.1:52614 - "POST /node/69ae52158ef4424e487b5134 HTTP/1.0" 202 Accepted 2026-03-11 04:52:51,610 - lava_callback - INFO - Starting async job submit for node 69ae52158ef4424e487b5134 2026-03-11 04:52:51,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:52:51,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52158ef4424e487b5134 HTTP/1.1" 200 None 2026-03-11 04:52:51,626 - lava_callback - DEBUG - Processing node 69ae52158ef4424e487b5134: baseline-arm-clabbe 2026-03-11 04:52:51,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae52158ef4424e487b5134 2026-03-11 04:52:51,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:52:51,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52158ef4424e487b5134 HTTP/1.1" 200 None 2026-03-11 04:52:51,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:52:51,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:52:51,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:52:51,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae52158ef4424e487b5134 HTTP/1.1" 200 None 2026-03-11 04:52:51,690 - lava_callback - ERROR - Error processing callback for node 69ae52158ef4424e487b5134: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45784 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:53:31,327 - lava_callback - INFO - Received callback for node 69ae523f8ef4424e487b5189 INFO: 172.21.0.1:54780 - "POST /node/69ae523f8ef4424e487b5189 HTTP/1.0" 202 Accepted 2026-03-11 04:53:31,333 - lava_callback - INFO - Starting async job submit for node 69ae523f8ef4424e487b5189 2026-03-11 04:53:31,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:53:31,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae523f8ef4424e487b5189 HTTP/1.1" 200 None 2026-03-11 04:53:31,349 - lava_callback - DEBUG - Processing node 69ae523f8ef4424e487b5189: baseline-arm-clabbe 2026-03-11 04:53:31,349 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae523f8ef4424e487b5189 2026-03-11 04:53:31,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:53:31,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae523f8ef4424e487b5189 HTTP/1.1" 200 None 2026-03-11 04:53:31,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:53:31,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:53:31,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:53:31,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae523f8ef4424e487b5189 HTTP/1.1" 200 None 2026-03-11 04:53:31,405 - lava_callback - ERROR - Error processing callback for node 69ae523f8ef4424e487b5189: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40618 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:54:11,388 - lava_callback - INFO - Received callback for node 69ae52698ef4424e487b51b3 INFO: 172.21.0.1:53124 - "POST /node/69ae52698ef4424e487b51b3 HTTP/1.0" 202 Accepted 2026-03-11 04:54:11,393 - lava_callback - INFO - Starting async job submit for node 69ae52698ef4424e487b51b3 2026-03-11 04:54:11,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:11,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52698ef4424e487b51b3 HTTP/1.1" 200 None 2026-03-11 04:54:11,409 - lava_callback - DEBUG - Processing node 69ae52698ef4424e487b51b3: baseline-arm-clabbe 2026-03-11 04:54:11,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae52698ef4424e487b51b3 2026-03-11 04:54:11,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:11,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52698ef4424e487b51b3 HTTP/1.1" 200 None 2026-03-11 04:54:11,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:11,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:54:11,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:11,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae52698ef4424e487b51b3 HTTP/1.1" 200 None 2026-03-11 04:54:11,482 - lava_callback - ERROR - Error processing callback for node 69ae52698ef4424e487b51b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33762 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:54:51,387 - lava_callback - INFO - Received callback for node 69ae52928ef4424e487b5205 INFO: 172.21.0.1:52862 - "POST /node/69ae52928ef4424e487b5205 HTTP/1.0" 202 Accepted 2026-03-11 04:54:51,393 - lava_callback - INFO - Starting async job submit for node 69ae52928ef4424e487b5205 2026-03-11 04:54:51,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:51,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52928ef4424e487b5205 HTTP/1.1" 200 None 2026-03-11 04:54:51,409 - lava_callback - DEBUG - Processing node 69ae52928ef4424e487b5205: baseline-arm-clabbe 2026-03-11 04:54:51,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae52928ef4424e487b5205 2026-03-11 04:54:51,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:51,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52928ef4424e487b5205 HTTP/1.1" 200 None 2026-03-11 04:54:51,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:51,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:54:51,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:54:51,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae52928ef4424e487b5205 HTTP/1.1" 200 None 2026-03-11 04:54:51,460 - lava_callback - ERROR - Error processing callback for node 69ae52928ef4424e487b5205: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35172 - "GET / HTTP/1.0" 200 OK 2026-03-11 04:55:31,462 - lava_callback - INFO - Received callback for node 69ae52bc8ef4424e487b5236 INFO: 172.21.0.1:38340 - "POST /node/69ae52bc8ef4424e487b5236 HTTP/1.0" 202 Accepted 2026-03-11 04:55:31,475 - lava_callback - INFO - Starting async job submit for node 69ae52bc8ef4424e487b5236 2026-03-11 04:55:31,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:55:31,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52bc8ef4424e487b5236 HTTP/1.1" 200 None 2026-03-11 04:55:31,510 - lava_callback - DEBUG - Processing node 69ae52bc8ef4424e487b5236: baseline-arm-clabbe 2026-03-11 04:55:31,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae52bc8ef4424e487b5236 2026-03-11 04:55:31,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:55:31,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae52bc8ef4424e487b5236 HTTP/1.1" 200 None 2026-03-11 04:55:31,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:55:31,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada13b8fc97e5ba07d4d75 HTTP/1.1" 200 None 2026-03-11 04:55:31,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 04:55:31,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae52bc8ef4424e487b5236 HTTP/1.1" 200 None 2026-03-11 04:55:31,581 - lava_callback - ERROR - Error processing callback for node 69ae52bc8ef4424e487b5236: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58998 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:02:43,389 - lava_callback - INFO - Received callback for node 69ae54618ef4424e487b52ef INFO: 172.21.0.1:58012 - "POST /node/69ae54618ef4424e487b52ef HTTP/1.0" 202 Accepted 2026-03-11 05:02:43,395 - lava_callback - INFO - Starting async job submit for node 69ae54618ef4424e487b52ef 2026-03-11 05:02:43,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:02:43,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54618ef4424e487b52ef HTTP/1.1" 200 None 2026-03-11 05:02:43,410 - lava_callback - DEBUG - Processing node 69ae54618ef4424e487b52ef: baseline-arm-baylibre 2026-03-11 05:02:43,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae54618ef4424e487b52ef 2026-03-11 05:02:43,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:02:43,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54618ef4424e487b52ef HTTP/1.1" 200 None 2026-03-11 05:02:43,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:02:43,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:02:43,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:02:43,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae54618ef4424e487b52ef HTTP/1.1" 200 None 2026-03-11 05:02:43,467 - lava_callback - ERROR - Error processing callback for node 69ae54618ef4424e487b52ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53876 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:03:31,791 - lava_callback - INFO - Received callback for node 69ae54908ef4424e487b5328 INFO: 172.21.0.1:46002 - "POST /node/69ae54908ef4424e487b5328 HTTP/1.0" 202 Accepted 2026-03-11 05:03:31,797 - lava_callback - INFO - Starting async job submit for node 69ae54908ef4424e487b5328 2026-03-11 05:03:31,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:03:31,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54908ef4424e487b5328 HTTP/1.1" 200 None 2026-03-11 05:03:31,813 - lava_callback - DEBUG - Processing node 69ae54908ef4424e487b5328: baseline-arm-clabbe 2026-03-11 05:03:31,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae54908ef4424e487b5328 2026-03-11 05:03:31,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:03:31,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54908ef4424e487b5328 HTTP/1.1" 200 None 2026-03-11 05:03:31,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:03:31,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:03:31,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:03:31,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae54908ef4424e487b5328 HTTP/1.1" 200 None 2026-03-11 05:03:31,875 - lava_callback - ERROR - Error processing callback for node 69ae54908ef4424e487b5328: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53106 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:04:11,748 - lava_callback - INFO - Received callback for node 69ae54b98ef4424e487b535f INFO: 172.21.0.1:33062 - "POST /node/69ae54b98ef4424e487b535f HTTP/1.0" 202 Accepted 2026-03-11 05:04:11,753 - lava_callback - INFO - Starting async job submit for node 69ae54b98ef4424e487b535f 2026-03-11 05:04:11,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:11,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54b98ef4424e487b535f HTTP/1.1" 200 None 2026-03-11 05:04:11,769 - lava_callback - DEBUG - Processing node 69ae54b98ef4424e487b535f: baseline-arm-clabbe 2026-03-11 05:04:11,769 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae54b98ef4424e487b535f 2026-03-11 05:04:11,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:11,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54b98ef4424e487b535f HTTP/1.1" 200 None 2026-03-11 05:04:11,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:11,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:04:11,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:11,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae54b98ef4424e487b535f HTTP/1.1" 200 None 2026-03-11 05:04:11,833 - lava_callback - ERROR - Error processing callback for node 69ae54b98ef4424e487b535f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58716 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:04:51,803 - lava_callback - INFO - Received callback for node 69ae54e48ef4424e487b5383 INFO: 172.21.0.1:36984 - "POST /node/69ae54e48ef4424e487b5383 HTTP/1.0" 202 Accepted 2026-03-11 05:04:51,809 - lava_callback - INFO - Starting async job submit for node 69ae54e48ef4424e487b5383 2026-03-11 05:04:51,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:51,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54e48ef4424e487b5383 HTTP/1.1" 200 None 2026-03-11 05:04:51,854 - lava_callback - DEBUG - Processing node 69ae54e48ef4424e487b5383: baseline-arm-clabbe 2026-03-11 05:04:51,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae54e48ef4424e487b5383 2026-03-11 05:04:51,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:51,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae54e48ef4424e487b5383 HTTP/1.1" 200 None 2026-03-11 05:04:51,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:51,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:04:51,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:04:51,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae54e48ef4424e487b5383 HTTP/1.1" 200 None 2026-03-11 05:04:51,930 - lava_callback - ERROR - Error processing callback for node 69ae54e48ef4424e487b5383: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46960 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:08:03,643 - lava_callback - INFO - Received callback for node 69ae55ab8ef4424e487b53ee INFO: 172.21.0.1:48310 - "POST /node/69ae55ab8ef4424e487b53ee HTTP/1.0" 202 Accepted 2026-03-11 05:08:03,649 - lava_callback - INFO - Starting async job submit for node 69ae55ab8ef4424e487b53ee 2026-03-11 05:08:03,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:08:03,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae55ab8ef4424e487b53ee HTTP/1.1" 200 None 2026-03-11 05:08:03,664 - lava_callback - DEBUG - Processing node 69ae55ab8ef4424e487b53ee: baseline-arm64-baylibre 2026-03-11 05:08:03,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae55ab8ef4424e487b53ee 2026-03-11 05:08:03,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:08:03,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae55ab8ef4424e487b53ee HTTP/1.1" 200 None 2026-03-11 05:08:03,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:08:03,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-11 05:08:03,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:08:03,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae55ab8ef4424e487b53ee HTTP/1.1" 200 None 2026-03-11 05:08:03,716 - lava_callback - ERROR - Error processing callback for node 69ae55ab8ef4424e487b53ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39972 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:09:32,051 - lava_callback - INFO - Received callback for node 69ae55ff8ef4424e487b5411 INFO: 172.21.0.1:44634 - "POST /node/69ae55ff8ef4424e487b5411 HTTP/1.0" 202 Accepted 2026-03-11 05:09:32,057 - lava_callback - INFO - Starting async job submit for node 69ae55ff8ef4424e487b5411 2026-03-11 05:09:32,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:09:32,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae55ff8ef4424e487b5411 HTTP/1.1" 200 None 2026-03-11 05:09:32,073 - lava_callback - DEBUG - Processing node 69ae55ff8ef4424e487b5411: baseline-arm64-clabbe 2026-03-11 05:09:32,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae55ff8ef4424e487b5411 2026-03-11 05:09:32,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:09:32,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae55ff8ef4424e487b5411 HTTP/1.1" 200 None 2026-03-11 05:09:32,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:09:32,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-11 05:09:32,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:09:32,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae55ff8ef4424e487b5411 HTTP/1.1" 200 None 2026-03-11 05:09:32,144 - lava_callback - ERROR - Error processing callback for node 69ae55ff8ef4424e487b5411: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42746 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:10:52,136 - lava_callback - INFO - Received callback for node 69ae564f8ef4424e487b5421 INFO: 172.21.0.1:42702 - "POST /node/69ae564f8ef4424e487b5421 HTTP/1.0" 202 Accepted 2026-03-11 05:10:52,141 - lava_callback - INFO - Starting async job submit for node 69ae564f8ef4424e487b5421 2026-03-11 05:10:52,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:10:52,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae564f8ef4424e487b5421 HTTP/1.1" 200 None 2026-03-11 05:10:52,183 - lava_callback - DEBUG - Processing node 69ae564f8ef4424e487b5421: baseline-arm64-clabbe 2026-03-11 05:10:52,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae564f8ef4424e487b5421 2026-03-11 05:10:52,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:10:53,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae564f8ef4424e487b5421 HTTP/1.1" 200 None 2026-03-11 05:10:53,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:10:53,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-11 05:10:53,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:10:53,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae564f8ef4424e487b5421 HTTP/1.1" 200 None 2026-03-11 05:10:53,458 - lava_callback - ERROR - Error processing callback for node 69ae564f8ef4424e487b5421: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40022 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:12:12,160 - lava_callback - INFO - Received callback for node 69ae56a38ef4424e487b5443 INFO: 172.21.0.1:59710 - "POST /node/69ae56a38ef4424e487b5443 HTTP/1.0" 202 Accepted 2026-03-11 05:12:12,165 - lava_callback - INFO - Starting async job submit for node 69ae56a38ef4424e487b5443 2026-03-11 05:12:12,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:12:12,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae56a38ef4424e487b5443 HTTP/1.1" 200 None 2026-03-11 05:12:12,182 - lava_callback - DEBUG - Processing node 69ae56a38ef4424e487b5443: baseline-arm64-clabbe 2026-03-11 05:12:12,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae56a38ef4424e487b5443 2026-03-11 05:12:12,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:12:12,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae56a38ef4424e487b5443 HTTP/1.1" 200 None 2026-03-11 05:12:12,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:12:12,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-11 05:12:12,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:12:12,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae56a38ef4424e487b5443 HTTP/1.1" 200 None 2026-03-11 05:12:12,233 - lava_callback - ERROR - Error processing callback for node 69ae56a38ef4424e487b5443: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51928 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:13:32,237 - lava_callback - INFO - Received callback for node 69ae56f38ef4424e487b545b INFO: 172.21.0.1:38302 - "POST /node/69ae56f38ef4424e487b545b HTTP/1.0" 202 Accepted 2026-03-11 05:13:32,242 - lava_callback - INFO - Starting async job submit for node 69ae56f38ef4424e487b545b 2026-03-11 05:13:32,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:13:32,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae56f38ef4424e487b545b HTTP/1.1" 200 None 2026-03-11 05:13:32,266 - lava_callback - DEBUG - Processing node 69ae56f38ef4424e487b545b: baseline-arm64-clabbe 2026-03-11 05:13:32,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae56f38ef4424e487b545b 2026-03-11 05:13:32,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:13:32,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae56f38ef4424e487b545b HTTP/1.1" 200 None 2026-03-11 05:13:32,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:13:32,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-11 05:13:32,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:13:32,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae56f38ef4424e487b545b HTTP/1.1" 200 None 2026-03-11 05:13:32,323 - lava_callback - ERROR - Error processing callback for node 69ae56f38ef4424e487b545b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:14:52,300 - lava_callback - INFO - Received callback for node 69ae57438ef4424e487b5486 INFO: 172.21.0.1:42076 - "POST /node/69ae57438ef4424e487b5486 HTTP/1.0" 202 Accepted 2026-03-11 05:14:52,306 - lava_callback - INFO - Starting async job submit for node 69ae57438ef4424e487b5486 2026-03-11 05:14:52,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:14:52,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57438ef4424e487b5486 HTTP/1.1" 200 None 2026-03-11 05:14:52,322 - lava_callback - DEBUG - Processing node 69ae57438ef4424e487b5486: baseline-arm64-clabbe 2026-03-11 05:14:52,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae57438ef4424e487b5486 2026-03-11 05:14:52,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:14:52,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57438ef4424e487b5486 HTTP/1.1" 200 None 2026-03-11 05:14:52,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:14:52,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-11 05:14:52,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:14:52,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae57438ef4424e487b5486 HTTP/1.1" 200 None 2026-03-11 05:14:52,378 - lava_callback - ERROR - Error processing callback for node 69ae57438ef4424e487b5486: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42718 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:16:04,327 - lava_callback - INFO - Received callback for node 69ae57808ef4424e487b5496 INFO: 172.21.0.1:42668 - "POST /node/69ae57808ef4424e487b5496 HTTP/1.0" 202 Accepted 2026-03-11 05:16:04,332 - lava_callback - INFO - Starting async job submit for node 69ae57808ef4424e487b5496 2026-03-11 05:16:04,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:04,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57808ef4424e487b5496 HTTP/1.1" 200 None 2026-03-11 05:16:04,348 - lava_callback - DEBUG - Processing node 69ae57808ef4424e487b5496: baseline-arm-baylibre 2026-03-11 05:16:04,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae57808ef4424e487b5496 2026-03-11 05:16:04,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:04,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57808ef4424e487b5496 HTTP/1.1" 200 None 2026-03-11 05:16:04,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:04,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:16:04,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:04,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae57808ef4424e487b5496 HTTP/1.1" 200 None 2026-03-11 05:16:04,444 - lava_callback - ERROR - Error processing callback for node 69ae57808ef4424e487b5496: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:16:52,342 - lava_callback - INFO - Received callback for node 69ae57ae8ef4424e487b54ba INFO: 172.21.0.1:42916 - "POST /node/69ae57ae8ef4424e487b54ba HTTP/1.0" 202 Accepted 2026-03-11 05:16:52,348 - lava_callback - INFO - Starting async job submit for node 69ae57ae8ef4424e487b54ba 2026-03-11 05:16:52,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:52,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57ae8ef4424e487b54ba HTTP/1.1" 200 None 2026-03-11 05:16:52,371 - lava_callback - DEBUG - Processing node 69ae57ae8ef4424e487b54ba: baseline-arm-clabbe 2026-03-11 05:16:52,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae57ae8ef4424e487b54ba 2026-03-11 05:16:52,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:52,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57ae8ef4424e487b54ba HTTP/1.1" 200 None 2026-03-11 05:16:52,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:52,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:16:52,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:16:52,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae57ae8ef4424e487b54ba HTTP/1.1" 200 None 2026-03-11 05:16:52,449 - lava_callback - ERROR - Error processing callback for node 69ae57ae8ef4424e487b54ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39518 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:17:32,345 - lava_callback - INFO - Received callback for node 69ae57d88ef4424e487b5500 INFO: 172.21.0.1:50372 - "POST /node/69ae57d88ef4424e487b5500 HTTP/1.0" 202 Accepted 2026-03-11 05:17:32,350 - lava_callback - INFO - Starting async job submit for node 69ae57d88ef4424e487b5500 2026-03-11 05:17:32,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:17:32,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57d88ef4424e487b5500 HTTP/1.1" 200 None 2026-03-11 05:17:32,367 - lava_callback - DEBUG - Processing node 69ae57d88ef4424e487b5500: baseline-arm-clabbe 2026-03-11 05:17:32,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae57d88ef4424e487b5500 2026-03-11 05:17:32,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:17:32,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae57d88ef4424e487b5500 HTTP/1.1" 200 None 2026-03-11 05:17:32,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:17:32,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:17:32,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:17:32,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae57d88ef4424e487b5500 HTTP/1.1" 200 None 2026-03-11 05:17:32,433 - lava_callback - ERROR - Error processing callback for node 69ae57d88ef4424e487b5500: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:18:12,397 - lava_callback - INFO - Received callback for node 69ae58028ef4424e487b552e INFO: 172.21.0.1:45458 - "POST /node/69ae58028ef4424e487b552e HTTP/1.0" 202 Accepted 2026-03-11 05:18:12,403 - lava_callback - INFO - Starting async job submit for node 69ae58028ef4424e487b552e 2026-03-11 05:18:12,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:18:12,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae58028ef4424e487b552e HTTP/1.1" 200 None 2026-03-11 05:18:12,419 - lava_callback - DEBUG - Processing node 69ae58028ef4424e487b552e: baseline-arm-clabbe 2026-03-11 05:18:12,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae58028ef4424e487b552e 2026-03-11 05:18:12,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:18:12,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae58028ef4424e487b552e HTTP/1.1" 200 None 2026-03-11 05:18:12,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:18:12,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:18:12,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:18:12,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae58028ef4424e487b552e HTTP/1.1" 200 None 2026-03-11 05:18:12,477 - lava_callback - ERROR - Error processing callback for node 69ae58028ef4424e487b552e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54002 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:24:32,723 - lava_callback - INFO - Received callback for node 69ae598c8ef4424e487b55e6 INFO: 172.21.0.1:58026 - "POST /node/69ae598c8ef4424e487b55e6 HTTP/1.0" 202 Accepted 2026-03-11 05:24:32,728 - lava_callback - INFO - Starting async job submit for node 69ae598c8ef4424e487b55e6 2026-03-11 05:24:32,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:24:32,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae598c8ef4424e487b55e6 HTTP/1.1" 200 None 2026-03-11 05:24:32,743 - lava_callback - DEBUG - Processing node 69ae598c8ef4424e487b55e6: baseline-arm-clabbe 2026-03-11 05:24:32,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae598c8ef4424e487b55e6 2026-03-11 05:24:32,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:24:32,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae598c8ef4424e487b55e6 HTTP/1.1" 200 None 2026-03-11 05:24:32,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:24:32,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:24:32,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:24:32,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae598c8ef4424e487b55e6 HTTP/1.1" 200 None 2026-03-11 05:24:32,807 - lava_callback - ERROR - Error processing callback for node 69ae598c8ef4424e487b55e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35774 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:25:32,739 - lava_callback - INFO - Received callback for node 69ae59b68ef4424e487b5605 INFO: 172.21.0.1:41144 - "POST /node/69ae59b68ef4424e487b5605 HTTP/1.0" 202 Accepted 2026-03-11 05:25:32,745 - lava_callback - INFO - Starting async job submit for node 69ae59b68ef4424e487b5605 2026-03-11 05:25:32,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:25:32,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae59b68ef4424e487b5605 HTTP/1.1" 200 None 2026-03-11 05:25:32,761 - lava_callback - DEBUG - Processing node 69ae59b68ef4424e487b5605: baseline-arm-clabbe 2026-03-11 05:25:32,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae59b68ef4424e487b5605 2026-03-11 05:25:32,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:25:32,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae59b68ef4424e487b5605 HTTP/1.1" 200 None 2026-03-11 05:25:32,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:25:32,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:25:32,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:25:32,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae59b68ef4424e487b5605 HTTP/1.1" 200 None 2026-03-11 05:25:32,827 - lava_callback - ERROR - Error processing callback for node 69ae59b68ef4424e487b5605: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34896 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:26:12,742 - lava_callback - INFO - Received callback for node 69ae59e18ef4424e487b564d INFO: 172.21.0.1:49484 - "POST /node/69ae59e18ef4424e487b564d HTTP/1.0" 202 Accepted 2026-03-11 05:26:12,748 - lava_callback - INFO - Starting async job submit for node 69ae59e18ef4424e487b564d 2026-03-11 05:26:12,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:12,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae59e18ef4424e487b564d HTTP/1.1" 200 None 2026-03-11 05:26:12,768 - lava_callback - DEBUG - Processing node 69ae59e18ef4424e487b564d: baseline-arm-clabbe 2026-03-11 05:26:12,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae59e18ef4424e487b564d 2026-03-11 05:26:12,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:12,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae59e18ef4424e487b564d HTTP/1.1" 200 None 2026-03-11 05:26:12,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:12,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aafe8b27cb11dcdfc758da HTTP/1.1" 200 None 2026-03-11 05:26:12,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:12,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae59e18ef4424e487b564d HTTP/1.1" 200 None 2026-03-11 05:26:12,837 - lava_callback - ERROR - Error processing callback for node 69ae59e18ef4424e487b564d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41296 - "GET / HTTP/1.0" 200 OK 2026-03-11 05:26:52,802 - lava_callback - INFO - Received callback for node 69ae5a198ef4424e487b568f INFO: 172.21.0.1:52948 - "POST /node/69ae5a198ef4424e487b568f HTTP/1.0" 202 Accepted 2026-03-11 05:26:52,808 - lava_callback - INFO - Starting async job submit for node 69ae5a198ef4424e487b568f 2026-03-11 05:26:52,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:52,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae5a198ef4424e487b568f HTTP/1.1" 200 None 2026-03-11 05:26:52,827 - lava_callback - DEBUG - Processing node 69ae5a198ef4424e487b568f: baseline-arm64-clabbe 2026-03-11 05:26:52,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae5a198ef4424e487b568f 2026-03-11 05:26:52,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:52,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae5a198ef4424e487b568f HTTP/1.1" 200 None 2026-03-11 05:26:52,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:52,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ada1488fc97e5ba07d4d8d HTTP/1.1" 200 None 2026-03-11 05:26:52,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 05:26:52,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae5a198ef4424e487b568f HTTP/1.1" 200 None 2026-03-11 05:26:52,883 - lava_callback - ERROR - Error processing callback for node 69ae5a198ef4424e487b568f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60198 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:44:28,888 - lava_callback - INFO - Received callback for node 69ae6c3a8ef4424e487b5a6f INFO: 172.21.0.1:60212 - "POST /node/69ae6c3a8ef4424e487b5a6f HTTP/1.0" 202 Accepted 2026-03-11 06:44:28,893 - lava_callback - INFO - Starting async job submit for node 69ae6c3a8ef4424e487b5a6f 2026-03-11 06:44:28,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:44:28,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6c3a8ef4424e487b5a6f HTTP/1.1" 200 None 2026-03-11 06:44:28,909 - lava_callback - DEBUG - Processing node 69ae6c3a8ef4424e487b5a6f: baseline-arm-baylibre 2026-03-11 06:44:28,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6c3a8ef4424e487b5a6f 2026-03-11 06:44:28,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:44:28,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6c3a8ef4424e487b5a6f HTTP/1.1" 200 None 2026-03-11 06:44:28,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:44:28,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:44:28,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:44:28,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6c3a8ef4424e487b5a6f HTTP/1.1" 200 None 2026-03-11 06:44:28,965 - lava_callback - ERROR - Error processing callback for node 69ae6c3a8ef4424e487b5a6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50732 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:45:08,847 - lava_callback - INFO - Received callback for node 69ae6c638ef4424e487b5abf INFO: 172.21.0.1:50736 - "POST /node/69ae6c638ef4424e487b5abf HTTP/1.0" 202 Accepted 2026-03-11 06:45:08,852 - lava_callback - INFO - Starting async job submit for node 69ae6c638ef4424e487b5abf 2026-03-11 06:45:08,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:08,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6c638ef4424e487b5abf HTTP/1.1" 200 None 2026-03-11 06:45:08,880 - lava_callback - DEBUG - Processing node 69ae6c638ef4424e487b5abf: baseline-arm-baylibre 2026-03-11 06:45:08,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6c638ef4424e487b5abf 2026-03-11 06:45:08,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:08,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6c638ef4424e487b5abf HTTP/1.1" 200 None 2026-03-11 06:45:08,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:08,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:45:08,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:08,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6c638ef4424e487b5abf HTTP/1.1" 200 None 2026-03-11 06:45:08,945 - lava_callback - ERROR - Error processing callback for node 69ae6c638ef4424e487b5abf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48372 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:45:48,875 - lava_callback - INFO - Received callback for node 69ae6c8b8ef4424e487b5ae3 INFO: 172.21.0.1:48386 - "POST /node/69ae6c8b8ef4424e487b5ae3 HTTP/1.0" 202 Accepted 2026-03-11 06:45:48,881 - lava_callback - INFO - Starting async job submit for node 69ae6c8b8ef4424e487b5ae3 2026-03-11 06:45:48,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:48,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6c8b8ef4424e487b5ae3 HTTP/1.1" 200 None 2026-03-11 06:45:48,900 - lava_callback - DEBUG - Processing node 69ae6c8b8ef4424e487b5ae3: baseline-arm-baylibre 2026-03-11 06:45:48,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6c8b8ef4424e487b5ae3 2026-03-11 06:45:48,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:48,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6c8b8ef4424e487b5ae3 HTTP/1.1" 200 None 2026-03-11 06:45:48,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:48,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:45:48,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:45:48,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6c8b8ef4424e487b5ae3 HTTP/1.1" 200 None 2026-03-11 06:45:48,987 - lava_callback - ERROR - Error processing callback for node 69ae6c8b8ef4424e487b5ae3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60440 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:46:28,921 - lava_callback - INFO - Received callback for node 69ae6cb48ef4424e487b5b32 INFO: 172.21.0.1:60448 - "POST /node/69ae6cb48ef4424e487b5b32 HTTP/1.0" 202 Accepted 2026-03-11 06:46:28,927 - lava_callback - INFO - Starting async job submit for node 69ae6cb48ef4424e487b5b32 2026-03-11 06:46:28,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:46:28,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6cb48ef4424e487b5b32 HTTP/1.1" 200 None 2026-03-11 06:46:28,944 - lava_callback - DEBUG - Processing node 69ae6cb48ef4424e487b5b32: baseline-arm-baylibre 2026-03-11 06:46:28,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6cb48ef4424e487b5b32 2026-03-11 06:46:28,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:46:28,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6cb48ef4424e487b5b32 HTTP/1.1" 200 None 2026-03-11 06:46:28,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:46:28,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:46:28,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:46:29,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6cb48ef4424e487b5b32 HTTP/1.1" 200 None 2026-03-11 06:46:29,013 - lava_callback - ERROR - Error processing callback for node 69ae6cb48ef4424e487b5b32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38436 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:47:09,021 - lava_callback - INFO - Received callback for node 69ae6cdd8ef4424e487b5b5c INFO: 172.21.0.1:38438 - "POST /node/69ae6cdd8ef4424e487b5b5c HTTP/1.0" 202 Accepted 2026-03-11 06:47:09,027 - lava_callback - INFO - Starting async job submit for node 69ae6cdd8ef4424e487b5b5c 2026-03-11 06:47:09,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:09,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6cdd8ef4424e487b5b5c HTTP/1.1" 200 None 2026-03-11 06:47:09,043 - lava_callback - DEBUG - Processing node 69ae6cdd8ef4424e487b5b5c: baseline-arm-baylibre 2026-03-11 06:47:09,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6cdd8ef4424e487b5b5c 2026-03-11 06:47:09,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:09,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6cdd8ef4424e487b5b5c HTTP/1.1" 200 None 2026-03-11 06:47:09,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:09,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:47:09,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:09,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6cdd8ef4424e487b5b5c HTTP/1.1" 200 None 2026-03-11 06:47:09,120 - lava_callback - ERROR - Error processing callback for node 69ae6cdd8ef4424e487b5b5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:47:48,975 - lava_callback - INFO - Received callback for node 69ae6d068ef4424e487b5b86 INFO: 172.21.0.1:42460 - "POST /node/69ae6d068ef4424e487b5b86 HTTP/1.0" 202 Accepted 2026-03-11 06:47:48,981 - lava_callback - INFO - Starting async job submit for node 69ae6d068ef4424e487b5b86 2026-03-11 06:47:48,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:49,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6d068ef4424e487b5b86 HTTP/1.1" 200 None 2026-03-11 06:47:49,011 - lava_callback - DEBUG - Processing node 69ae6d068ef4424e487b5b86: baseline-arm-baylibre 2026-03-11 06:47:49,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6d068ef4424e487b5b86 2026-03-11 06:47:49,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:49,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6d068ef4424e487b5b86 HTTP/1.1" 200 None 2026-03-11 06:47:49,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:49,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:47:49,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:47:49,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6d068ef4424e487b5b86 HTTP/1.1" 200 None 2026-03-11 06:47:49,077 - lava_callback - ERROR - Error processing callback for node 69ae6d068ef4424e487b5b86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46390 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:49:56,178 - lava_callback - INFO - Received callback for node 69ae6d7f8ef4424e487b5c2c INFO: 172.21.0.1:34466 - "POST /node/69ae6d7f8ef4424e487b5c2c HTTP/1.0" 202 Accepted 2026-03-11 06:49:56,183 - lava_callback - INFO - Starting async job submit for node 69ae6d7f8ef4424e487b5c2c 2026-03-11 06:49:56,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:49:56,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6d7f8ef4424e487b5c2c HTTP/1.1" 200 None 2026-03-11 06:49:56,199 - lava_callback - DEBUG - Processing node 69ae6d7f8ef4424e487b5c2c: baseline-arm-clabbe 2026-03-11 06:49:56,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6d7f8ef4424e487b5c2c 2026-03-11 06:49:56,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:49:56,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6d7f8ef4424e487b5c2c HTTP/1.1" 200 None 2026-03-11 06:49:56,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:49:56,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:49:56,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:49:56,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6d7f8ef4424e487b5c2c HTTP/1.1" 200 None 2026-03-11 06:49:56,268 - lava_callback - ERROR - Error processing callback for node 69ae6d7f8ef4424e487b5c2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36732 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:50:36,111 - lava_callback - INFO - Received callback for node 69ae6da88ef4424e487b5c76 INFO: 172.21.0.1:36748 - "POST /node/69ae6da88ef4424e487b5c76 HTTP/1.0" 202 Accepted 2026-03-11 06:50:36,117 - lava_callback - INFO - Starting async job submit for node 69ae6da88ef4424e487b5c76 2026-03-11 06:50:36,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:50:36,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6da88ef4424e487b5c76 HTTP/1.1" 200 None 2026-03-11 06:50:36,133 - lava_callback - DEBUG - Processing node 69ae6da88ef4424e487b5c76: baseline-arm-clabbe 2026-03-11 06:50:36,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6da88ef4424e487b5c76 2026-03-11 06:50:36,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:50:36,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6da88ef4424e487b5c76 HTTP/1.1" 200 None 2026-03-11 06:50:36,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:50:36,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:50:36,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:50:36,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6da88ef4424e487b5c76 HTTP/1.1" 200 None 2026-03-11 06:50:36,199 - lava_callback - ERROR - Error processing callback for node 69ae6da88ef4424e487b5c76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43316 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-11 06:51:16,189 - lava_callback - INFO - Received callback for node 69ae6dd18ef4424e487b5ca4 INFO: 172.21.0.1:43322 - "POST /node/69ae6dd18ef4424e487b5ca4 HTTP/1.0" 202 Accepted 2026-03-11 06:51:16,194 - lava_callback - INFO - Starting async job submit for node 69ae6dd18ef4424e487b5ca4 2026-03-11 06:51:16,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:16,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6dd18ef4424e487b5ca4 HTTP/1.1" 200 None 2026-03-11 06:51:16,211 - lava_callback - DEBUG - Processing node 69ae6dd18ef4424e487b5ca4: baseline-arm-clabbe 2026-03-11 06:51:16,211 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6dd18ef4424e487b5ca4 2026-03-11 06:51:16,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:16,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6dd18ef4424e487b5ca4 HTTP/1.1" 200 None 2026-03-11 06:51:16,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:16,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:51:16,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:16,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6dd18ef4424e487b5ca4 HTTP/1.1" 200 None 2026-03-11 06:51:16,277 - lava_callback - ERROR - Error processing callback for node 69ae6dd18ef4424e487b5ca4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43990 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:51:56,167 - lava_callback - INFO - Received callback for node 69ae6dfa8ef4424e487b5ccf INFO: 172.21.0.1:45324 - "POST /node/69ae6dfa8ef4424e487b5ccf HTTP/1.0" 202 Accepted 2026-03-11 06:51:56,172 - lava_callback - INFO - Starting async job submit for node 69ae6dfa8ef4424e487b5ccf 2026-03-11 06:51:56,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:56,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6dfa8ef4424e487b5ccf HTTP/1.1" 200 None 2026-03-11 06:51:56,189 - lava_callback - DEBUG - Processing node 69ae6dfa8ef4424e487b5ccf: baseline-arm-clabbe 2026-03-11 06:51:56,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6dfa8ef4424e487b5ccf 2026-03-11 06:51:56,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:56,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6dfa8ef4424e487b5ccf HTTP/1.1" 200 None 2026-03-11 06:51:56,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:56,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:51:56,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:51:56,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6dfa8ef4424e487b5ccf HTTP/1.1" 200 None 2026-03-11 06:51:56,250 - lava_callback - ERROR - Error processing callback for node 69ae6dfa8ef4424e487b5ccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42472 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:52:36,242 - lava_callback - INFO - Received callback for node 69ae6e238ef4424e487b5d20 INFO: 172.21.0.1:42476 - "POST /node/69ae6e238ef4424e487b5d20 HTTP/1.0" 202 Accepted 2026-03-11 06:52:36,248 - lava_callback - INFO - Starting async job submit for node 69ae6e238ef4424e487b5d20 2026-03-11 06:52:36,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:52:36,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e238ef4424e487b5d20 HTTP/1.1" 200 None 2026-03-11 06:52:36,264 - lava_callback - DEBUG - Processing node 69ae6e238ef4424e487b5d20: baseline-arm-clabbe 2026-03-11 06:52:36,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6e238ef4424e487b5d20 2026-03-11 06:52:36,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:52:36,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e238ef4424e487b5d20 HTTP/1.1" 200 None 2026-03-11 06:52:36,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:52:36,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11668ef4424e487b2ea2 HTTP/1.1" 200 None 2026-03-11 06:52:36,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:52:36,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6e238ef4424e487b5d20 HTTP/1.1" 200 None 2026-03-11 06:52:36,340 - lava_callback - ERROR - Error processing callback for node 69ae6e238ef4424e487b5d20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:53:09,305 - lava_callback - INFO - Received callback for node 69ae6e478ef4424e487b5d44 INFO: 172.21.0.1:33366 - "POST /node/69ae6e478ef4424e487b5d44 HTTP/1.0" 202 Accepted 2026-03-11 06:53:09,310 - lava_callback - INFO - Starting async job submit for node 69ae6e478ef4424e487b5d44 2026-03-11 06:53:09,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:09,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e478ef4424e487b5d44 HTTP/1.1" 200 None 2026-03-11 06:53:09,327 - lava_callback - DEBUG - Processing node 69ae6e478ef4424e487b5d44: baseline-arm-baylibre 2026-03-11 06:53:09,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6e478ef4424e487b5d44 2026-03-11 06:53:09,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:09,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e478ef4424e487b5d44 HTTP/1.1" 200 None 2026-03-11 06:53:09,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:09,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:53:09,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:09,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6e478ef4424e487b5d44 HTTP/1.1" 200 None 2026-03-11 06:53:09,384 - lava_callback - ERROR - Error processing callback for node 69ae6e478ef4424e487b5d44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40112 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:53:49,295 - lava_callback - INFO - Received callback for node 69ae6e6f8ef4424e487b5d69 INFO: 172.21.0.1:40116 - "POST /node/69ae6e6f8ef4424e487b5d69 HTTP/1.0" 202 Accepted 2026-03-11 06:53:49,301 - lava_callback - INFO - Starting async job submit for node 69ae6e6f8ef4424e487b5d69 2026-03-11 06:53:49,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:49,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e6f8ef4424e487b5d69 HTTP/1.1" 200 None 2026-03-11 06:53:49,316 - lava_callback - DEBUG - Processing node 69ae6e6f8ef4424e487b5d69: baseline-arm-baylibre 2026-03-11 06:53:49,316 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6e6f8ef4424e487b5d69 2026-03-11 06:53:49,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:49,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e6f8ef4424e487b5d69 HTTP/1.1" 200 None 2026-03-11 06:53:49,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:49,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:53:49,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:53:49,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6e6f8ef4424e487b5d69 HTTP/1.1" 200 None 2026-03-11 06:53:49,379 - lava_callback - ERROR - Error processing callback for node 69ae6e6f8ef4424e487b5d69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55146 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:54:29,298 - lava_callback - INFO - Received callback for node 69ae6e998ef4424e487b5db2 INFO: 172.21.0.1:55156 - "POST /node/69ae6e998ef4424e487b5db2 HTTP/1.0" 202 Accepted 2026-03-11 06:54:29,304 - lava_callback - INFO - Starting async job submit for node 69ae6e998ef4424e487b5db2 2026-03-11 06:54:29,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:54:29,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e998ef4424e487b5db2 HTTP/1.1" 200 None 2026-03-11 06:54:29,320 - lava_callback - DEBUG - Processing node 69ae6e998ef4424e487b5db2: baseline-arm-baylibre 2026-03-11 06:54:29,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6e998ef4424e487b5db2 2026-03-11 06:54:29,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:54:29,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6e998ef4424e487b5db2 HTTP/1.1" 200 None 2026-03-11 06:54:29,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:54:29,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:54:29,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:54:29,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6e998ef4424e487b5db2 HTTP/1.1" 200 None 2026-03-11 06:54:29,389 - lava_callback - ERROR - Error processing callback for node 69ae6e998ef4424e487b5db2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58478 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:55:09,352 - lava_callback - INFO - Received callback for node 69ae6ec38ef4424e487b5de3 INFO: 172.21.0.1:58494 - "POST /node/69ae6ec38ef4424e487b5de3 HTTP/1.0" 202 Accepted 2026-03-11 06:55:09,358 - lava_callback - INFO - Starting async job submit for node 69ae6ec38ef4424e487b5de3 2026-03-11 06:55:09,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:09,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6ec38ef4424e487b5de3 HTTP/1.1" 200 None 2026-03-11 06:55:09,374 - lava_callback - DEBUG - Processing node 69ae6ec38ef4424e487b5de3: baseline-arm-baylibre 2026-03-11 06:55:09,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6ec38ef4424e487b5de3 2026-03-11 06:55:09,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:09,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6ec38ef4424e487b5de3 HTTP/1.1" 200 None 2026-03-11 06:55:09,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:09,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:55:09,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:09,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6ec38ef4424e487b5de3 HTTP/1.1" 200 None 2026-03-11 06:55:09,438 - lava_callback - ERROR - Error processing callback for node 69ae6ec38ef4424e487b5de3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36916 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:55:49,432 - lava_callback - INFO - Received callback for node 69ae6eeb8ef4424e487b5e0c INFO: 172.21.0.1:36924 - "POST /node/69ae6eeb8ef4424e487b5e0c HTTP/1.0" 202 Accepted 2026-03-11 06:55:49,437 - lava_callback - INFO - Starting async job submit for node 69ae6eeb8ef4424e487b5e0c 2026-03-11 06:55:49,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:49,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6eeb8ef4424e487b5e0c HTTP/1.1" 200 None 2026-03-11 06:55:49,461 - lava_callback - DEBUG - Processing node 69ae6eeb8ef4424e487b5e0c: baseline-arm-baylibre 2026-03-11 06:55:49,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6eeb8ef4424e487b5e0c 2026-03-11 06:55:49,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:49,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6eeb8ef4424e487b5e0c HTTP/1.1" 200 None 2026-03-11 06:55:49,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:49,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:55:49,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:55:49,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6eeb8ef4424e487b5e0c HTTP/1.1" 200 None 2026-03-11 06:55:49,513 - lava_callback - ERROR - Error processing callback for node 69ae6eeb8ef4424e487b5e0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:57:36,464 - lava_callback - INFO - Received callback for node 69ae6f558ef4424e487b5e77 INFO: 172.21.0.1:47612 - "POST /node/69ae6f558ef4424e487b5e77 HTTP/1.0" 202 Accepted 2026-03-11 06:57:36,469 - lava_callback - INFO - Starting async job submit for node 69ae6f558ef4424e487b5e77 2026-03-11 06:57:36,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:57:36,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6f558ef4424e487b5e77 HTTP/1.1" 200 None 2026-03-11 06:57:36,495 - lava_callback - DEBUG - Processing node 69ae6f558ef4424e487b5e77: baseline-arm-clabbe 2026-03-11 06:57:36,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6f558ef4424e487b5e77 2026-03-11 06:57:36,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:57:36,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6f558ef4424e487b5e77 HTTP/1.1" 200 None 2026-03-11 06:57:36,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:57:36,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:57:36,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:57:36,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6f558ef4424e487b5e77 HTTP/1.1" 200 None 2026-03-11 06:57:36,551 - lava_callback - ERROR - Error processing callback for node 69ae6f558ef4424e487b5e77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58608 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:59:09,924 - lava_callback - INFO - Received callback for node 69ae6fb68ef4424e487b5ec1 INFO: 172.21.0.1:37428 - "POST /node/69ae6fb68ef4424e487b5ec1 HTTP/1.0" 202 Accepted 2026-03-11 06:59:09,930 - lava_callback - INFO - Starting async job submit for node 69ae6fb68ef4424e487b5ec1 2026-03-11 06:59:09,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:09,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fb68ef4424e487b5ec1 HTTP/1.1" 200 None 2026-03-11 06:59:09,945 - lava_callback - DEBUG - Processing node 69ae6fb68ef4424e487b5ec1: baseline-arm-baylibre 2026-03-11 06:59:09,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6fb68ef4424e487b5ec1 2026-03-11 06:59:09,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:09,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fb68ef4424e487b5ec1 HTTP/1.1" 200 None 2026-03-11 06:59:09,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:09,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:59:09,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:09,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6fb68ef4424e487b5ec1 HTTP/1.1" 200 None 2026-03-11 06:59:09,999 - lava_callback - ERROR - Error processing callback for node 69ae6fb68ef4424e487b5ec1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK 2026-03-11 06:59:57,520 - lava_callback - INFO - Received callback for node 69ae6fe38ef4424e487b5efb INFO: 172.21.0.1:46994 - "POST /node/69ae6fe38ef4424e487b5efb HTTP/1.0" 202 Accepted 2026-03-11 06:59:57,526 - lava_callback - INFO - Starting async job submit for node 69ae6fe38ef4424e487b5efb 2026-03-11 06:59:57,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:57,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fe38ef4424e487b5efb HTTP/1.1" 200 None 2026-03-11 06:59:57,541 - lava_callback - DEBUG - Processing node 69ae6fe38ef4424e487b5efb: baseline-arm-clabbe 2026-03-11 06:59:57,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae6fe38ef4424e487b5efb 2026-03-11 06:59:57,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:57,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fe38ef4424e487b5efb HTTP/1.1" 200 None 2026-03-11 06:59:57,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:57,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 06:59:57,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 06:59:57,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6fe38ef4424e487b5efb HTTP/1.1" 200 None 2026-03-11 06:59:57,596 - lava_callback - ERROR - Error processing callback for node 69ae6fe38ef4424e487b5efb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59078 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:00:36,502 - lava_callback - INFO - Received callback for node 69ae700c8ef4424e487b5f1e INFO: 172.21.0.1:59088 - "POST /node/69ae700c8ef4424e487b5f1e HTTP/1.0" 202 Accepted 2026-03-11 07:00:36,509 - lava_callback - INFO - Starting async job submit for node 69ae700c8ef4424e487b5f1e 2026-03-11 07:00:36,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:00:36,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae700c8ef4424e487b5f1e HTTP/1.1" 200 None 2026-03-11 07:00:36,525 - lava_callback - DEBUG - Processing node 69ae700c8ef4424e487b5f1e: baseline-arm-clabbe 2026-03-11 07:00:36,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae700c8ef4424e487b5f1e 2026-03-11 07:00:36,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:59102 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:00:36,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae700c8ef4424e487b5f1e HTTP/1.1" 200 None 2026-03-11 07:00:36,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:00:36,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:00:36,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:00:36,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae700c8ef4424e487b5f1e HTTP/1.1" 200 None 2026-03-11 07:00:36,586 - lava_callback - ERROR - Error processing callback for node 69ae700c8ef4424e487b5f1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57204 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:01:09,718 - lava_callback - INFO - Received callback for node 69ae702f8ef4424e487b5f45 INFO: 172.21.0.1:40780 - "POST /node/69ae702f8ef4424e487b5f45 HTTP/1.0" 202 Accepted 2026-03-11 07:01:09,724 - lava_callback - INFO - Starting async job submit for node 69ae702f8ef4424e487b5f45 2026-03-11 07:01:09,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:09,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae702f8ef4424e487b5f45 HTTP/1.1" 200 None 2026-03-11 07:01:09,744 - lava_callback - DEBUG - Processing node 69ae702f8ef4424e487b5f45: baseline-arm-baylibre 2026-03-11 07:01:09,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae702f8ef4424e487b5f45 2026-03-11 07:01:09,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:09,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae702f8ef4424e487b5f45 HTTP/1.1" 200 None 2026-03-11 07:01:09,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:09,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:01:09,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:09,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae702f8ef4424e487b5f45 HTTP/1.1" 200 None 2026-03-11 07:01:09,811 - lava_callback - ERROR - Error processing callback for node 69ae702f8ef4424e487b5f45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:01:49,685 - lava_callback - INFO - Received callback for node 69ae70588ef4424e487b5f92 INFO: 172.21.0.1:57290 - "POST /node/69ae70588ef4424e487b5f92 HTTP/1.0" 202 Accepted 2026-03-11 07:01:49,691 - lava_callback - INFO - Starting async job submit for node 69ae70588ef4424e487b5f92 2026-03-11 07:01:49,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70588ef4424e487b5f92 HTTP/1.1" 200 None 2026-03-11 07:01:49,715 - lava_callback - DEBUG - Processing node 69ae70588ef4424e487b5f92: baseline-arm-baylibre 2026-03-11 07:01:49,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae70588ef4424e487b5f92 2026-03-11 07:01:49,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:49,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70588ef4424e487b5f92 HTTP/1.1" 200 None 2026-03-11 07:01:49,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:49,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:01:49,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:01:49,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae70588ef4424e487b5f92 HTTP/1.1" 200 None 2026-03-11 07:01:49,768 - lava_callback - ERROR - Error processing callback for node 69ae70588ef4424e487b5f92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56604 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:02:29,757 - lava_callback - INFO - Received callback for node 69ae70818ef4424e487b5fb0 INFO: 172.21.0.1:44742 - "POST /node/69ae70818ef4424e487b5fb0 HTTP/1.0" 202 Accepted 2026-03-11 07:02:29,762 - lava_callback - INFO - Starting async job submit for node 69ae70818ef4424e487b5fb0 2026-03-11 07:02:29,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:02:29,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70818ef4424e487b5fb0 HTTP/1.1" 200 None 2026-03-11 07:02:29,785 - lava_callback - DEBUG - Processing node 69ae70818ef4424e487b5fb0: baseline-arm-baylibre 2026-03-11 07:02:29,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae70818ef4424e487b5fb0 2026-03-11 07:02:29,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:02:29,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70818ef4424e487b5fb0 HTTP/1.1" 200 None 2026-03-11 07:02:29,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:02:29,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:02:29,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:02:29,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae70818ef4424e487b5fb0 HTTP/1.1" 200 None 2026-03-11 07:02:29,844 - lava_callback - ERROR - Error processing callback for node 69ae70818ef4424e487b5fb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:03:09,773 - lava_callback - INFO - Received callback for node 69ae70a98ef4424e487b5fd0 INFO: 172.21.0.1:39394 - "POST /node/69ae70a98ef4424e487b5fd0 HTTP/1.0" 202 Accepted 2026-03-11 07:03:09,778 - lava_callback - INFO - Starting async job submit for node 69ae70a98ef4424e487b5fd0 2026-03-11 07:03:09,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:09,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70a98ef4424e487b5fd0 HTTP/1.1" 200 None 2026-03-11 07:03:09,797 - lava_callback - DEBUG - Processing node 69ae70a98ef4424e487b5fd0: baseline-arm-baylibre 2026-03-11 07:03:09,797 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae70a98ef4424e487b5fd0 2026-03-11 07:03:09,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:09,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70a98ef4424e487b5fd0 HTTP/1.1" 200 None 2026-03-11 07:03:09,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:09,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:03:09,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:09,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae70a98ef4424e487b5fd0 HTTP/1.1" 200 None 2026-03-11 07:03:09,860 - lava_callback - ERROR - Error processing callback for node 69ae70a98ef4424e487b5fd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53854 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:03:56,715 - lava_callback - INFO - Received callback for node 69ae70d38ef4424e487b601b INFO: 172.21.0.1:53866 - "POST /node/69ae70d38ef4424e487b601b HTTP/1.0" 202 Accepted 2026-03-11 07:03:56,720 - lava_callback - INFO - Starting async job submit for node 69ae70d38ef4424e487b601b 2026-03-11 07:03:56,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:56,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70d38ef4424e487b601b HTTP/1.1" 200 None 2026-03-11 07:03:56,736 - lava_callback - DEBUG - Processing node 69ae70d38ef4424e487b601b: baseline-arm-clabbe 2026-03-11 07:03:56,736 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae70d38ef4424e487b601b 2026-03-11 07:03:56,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:56,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70d38ef4424e487b601b HTTP/1.1" 200 None 2026-03-11 07:03:56,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:56,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:03:56,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:03:56,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae70d38ef4424e487b601b HTTP/1.1" 200 None 2026-03-11 07:03:56,788 - lava_callback - ERROR - Error processing callback for node 69ae70d38ef4424e487b601b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:04:36,681 - lava_callback - INFO - Received callback for node 69ae70fb8ef4424e487b6048 INFO: 172.21.0.1:60342 - "POST /node/69ae70fb8ef4424e487b6048 HTTP/1.0" 202 Accepted 2026-03-11 07:04:36,687 - lava_callback - INFO - Starting async job submit for node 69ae70fb8ef4424e487b6048 2026-03-11 07:04:36,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:04:36,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70fb8ef4424e487b6048 HTTP/1.1" 200 None 2026-03-11 07:04:36,703 - lava_callback - DEBUG - Processing node 69ae70fb8ef4424e487b6048: baseline-arm-clabbe 2026-03-11 07:04:36,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae70fb8ef4424e487b6048 2026-03-11 07:04:36,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:04:36,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae70fb8ef4424e487b6048 HTTP/1.1" 200 None 2026-03-11 07:04:36,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:04:36,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:04:36,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:04:36,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae70fb8ef4424e487b6048 HTTP/1.1" 200 None 2026-03-11 07:04:36,757 - lava_callback - ERROR - Error processing callback for node 69ae70fb8ef4424e487b6048: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:05:16,744 - lava_callback - INFO - Received callback for node 69ae71238ef4424e487b6066 INFO: 172.21.0.1:38464 - "POST /node/69ae71238ef4424e487b6066 HTTP/1.0" 202 Accepted 2026-03-11 07:05:16,750 - lava_callback - INFO - Starting async job submit for node 69ae71238ef4424e487b6066 2026-03-11 07:05:16,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:05:16,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae71238ef4424e487b6066 HTTP/1.1" 200 None 2026-03-11 07:05:16,766 - lava_callback - DEBUG - Processing node 69ae71238ef4424e487b6066: baseline-arm-clabbe 2026-03-11 07:05:16,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae71238ef4424e487b6066 2026-03-11 07:05:16,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:05:16,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae71238ef4424e487b6066 HTTP/1.1" 200 None 2026-03-11 07:05:16,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:05:16,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-11 07:05:16,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:05:16,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae71238ef4424e487b6066 HTTP/1.1" 200 None 2026-03-11 07:05:16,823 - lava_callback - ERROR - Error processing callback for node 69ae71238ef4424e487b6066: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45336 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:08:30,117 - lava_callback - INFO - Received callback for node 69ae71ec8ef4424e487b60c7 INFO: 172.21.0.1:55818 - "POST /node/69ae71ec8ef4424e487b60c7 HTTP/1.0" 202 Accepted 2026-03-11 07:08:30,123 - lava_callback - INFO - Starting async job submit for node 69ae71ec8ef4424e487b60c7 2026-03-11 07:08:30,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:08:30,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae71ec8ef4424e487b60c7 HTTP/1.1" 200 None 2026-03-11 07:08:30,143 - lava_callback - DEBUG - Processing node 69ae71ec8ef4424e487b60c7: baseline-arm64-baylibre 2026-03-11 07:08:30,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae71ec8ef4424e487b60c7 2026-03-11 07:08:30,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:08:30,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae71ec8ef4424e487b60c7 HTTP/1.1" 200 None 2026-03-11 07:08:30,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:08:30,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 07:08:30,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:08:30,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae71ec8ef4424e487b60c7 HTTP/1.1" 200 None 2026-03-11 07:08:30,200 - lava_callback - ERROR - Error processing callback for node 69ae71ec8ef4424e487b60c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47000 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:10:10,235 - lava_callback - INFO - Received callback for node 69ae724d8ef4424e487b60e9 INFO: 172.21.0.1:59432 - "POST /node/69ae724d8ef4424e487b60e9 HTTP/1.0" 202 Accepted 2026-03-11 07:10:10,240 - lava_callback - INFO - Starting async job submit for node 69ae724d8ef4424e487b60e9 2026-03-11 07:10:10,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:10:10,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae724d8ef4424e487b60e9 HTTP/1.1" 200 None 2026-03-11 07:10:10,257 - lava_callback - DEBUG - Processing node 69ae724d8ef4424e487b60e9: baseline-arm64-baylibre 2026-03-11 07:10:10,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae724d8ef4424e487b60e9 2026-03-11 07:10:10,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:10:10,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae724d8ef4424e487b60e9 HTTP/1.1" 200 None 2026-03-11 07:10:10,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:10:10,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 07:10:10,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:10:10,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae724d8ef4424e487b60e9 HTTP/1.1" 200 None 2026-03-11 07:10:10,315 - lava_callback - ERROR - Error processing callback for node 69ae724d8ef4424e487b60e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45952 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:11:30,262 - lava_callback - INFO - Received callback for node 69ae729a8ef4424e487b6105 INFO: 172.21.0.1:47808 - "POST /node/69ae729a8ef4424e487b6105 HTTP/1.0" 202 Accepted 2026-03-11 07:11:30,267 - lava_callback - INFO - Starting async job submit for node 69ae729a8ef4424e487b6105 2026-03-11 07:11:30,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:11:30,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae729a8ef4424e487b6105 HTTP/1.1" 200 None 2026-03-11 07:11:30,283 - lava_callback - DEBUG - Processing node 69ae729a8ef4424e487b6105: baseline-arm64-baylibre 2026-03-11 07:11:30,284 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae729a8ef4424e487b6105 2026-03-11 07:11:30,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:11:30,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae729a8ef4424e487b6105 HTTP/1.1" 200 None 2026-03-11 07:11:30,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:11:30,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 07:11:30,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:11:30,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae729a8ef4424e487b6105 HTTP/1.1" 200 None 2026-03-11 07:11:30,340 - lava_callback - ERROR - Error processing callback for node 69ae729a8ef4424e487b6105: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55386 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:12:57,062 - lava_callback - INFO - Received callback for node 69ae72ec8ef4424e487b611e INFO: 172.21.0.1:55402 - "POST /node/69ae72ec8ef4424e487b611e HTTP/1.0" 202 Accepted 2026-03-11 07:12:57,068 - lava_callback - INFO - Starting async job submit for node 69ae72ec8ef4424e487b611e 2026-03-11 07:12:57,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:12:57,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae72ec8ef4424e487b611e HTTP/1.1" 200 None 2026-03-11 07:12:57,100 - lava_callback - DEBUG - Processing node 69ae72ec8ef4424e487b611e: baseline-arm64-clabbe 2026-03-11 07:12:57,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae72ec8ef4424e487b611e 2026-03-11 07:12:57,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:12:57,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae72ec8ef4424e487b611e HTTP/1.1" 200 None 2026-03-11 07:12:57,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:12:57,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 07:12:57,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:12:57,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae72ec8ef4424e487b611e HTTP/1.1" 200 None 2026-03-11 07:12:57,185 - lava_callback - ERROR - Error processing callback for node 69ae72ec8ef4424e487b611e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50936 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:14:17,079 - lava_callback - INFO - Received callback for node 69ae73398ef4424e487b613c INFO: 172.21.0.1:50942 - "POST /node/69ae73398ef4424e487b613c HTTP/1.0" 202 Accepted 2026-03-11 07:14:17,085 - lava_callback - INFO - Starting async job submit for node 69ae73398ef4424e487b613c 2026-03-11 07:14:17,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:14:17,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73398ef4424e487b613c HTTP/1.1" 200 None 2026-03-11 07:14:17,102 - lava_callback - DEBUG - Processing node 69ae73398ef4424e487b613c: baseline-arm64-clabbe 2026-03-11 07:14:17,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae73398ef4424e487b613c 2026-03-11 07:14:17,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:14:17,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73398ef4424e487b613c HTTP/1.1" 200 None 2026-03-11 07:14:17,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:14:17,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 07:14:17,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:14:17,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae73398ef4424e487b613c HTTP/1.1" 200 None 2026-03-11 07:14:17,177 - lava_callback - ERROR - Error processing callback for node 69ae73398ef4424e487b613c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58948 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:15:17,133 - lava_callback - INFO - Received callback for node 69ae737a8ef4424e487b6164 INFO: 172.21.0.1:58960 - "POST /node/69ae737a8ef4424e487b6164 HTTP/1.0" 202 Accepted 2026-03-11 07:15:17,138 - lava_callback - INFO - Starting async job submit for node 69ae737a8ef4424e487b6164 2026-03-11 07:15:17,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:17,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae737a8ef4424e487b6164 HTTP/1.1" 200 None 2026-03-11 07:15:17,161 - lava_callback - DEBUG - Processing node 69ae737a8ef4424e487b6164: baseline-arm-clabbe 2026-03-11 07:15:17,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae737a8ef4424e487b6164 2026-03-11 07:15:17,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:17,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae737a8ef4424e487b6164 HTTP/1.1" 200 None 2026-03-11 07:15:17,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:17,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 07:15:17,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:17,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae737a8ef4424e487b6164 HTTP/1.1" 200 None 2026-03-11 07:15:17,221 - lava_callback - ERROR - Error processing callback for node 69ae737a8ef4424e487b6164: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40748 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:15:57,146 - lava_callback - INFO - Received callback for node 69ae73a78ef4424e487b61b7 INFO: 172.21.0.1:40754 - "POST /node/69ae73a78ef4424e487b61b7 HTTP/1.0" 202 Accepted 2026-03-11 07:15:57,152 - lava_callback - INFO - Starting async job submit for node 69ae73a78ef4424e487b61b7 2026-03-11 07:15:57,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:57,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73a78ef4424e487b61b7 HTTP/1.1" 200 None 2026-03-11 07:15:57,168 - lava_callback - DEBUG - Processing node 69ae73a78ef4424e487b61b7: baseline-arm-clabbe 2026-03-11 07:15:57,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae73a78ef4424e487b61b7 2026-03-11 07:15:57,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:57,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73a78ef4424e487b61b7 HTTP/1.1" 200 None 2026-03-11 07:15:57,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:57,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 07:15:57,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:15:57,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae73a78ef4424e487b61b7 HTTP/1.1" 200 None 2026-03-11 07:15:57,221 - lava_callback - ERROR - Error processing callback for node 69ae73a78ef4424e487b61b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57986 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:16:37,217 - lava_callback - INFO - Received callback for node 69ae73cf8ef4424e487b61dc INFO: 172.21.0.1:58002 - "POST /node/69ae73cf8ef4424e487b61dc HTTP/1.0" 202 Accepted 2026-03-11 07:16:37,223 - lava_callback - INFO - Starting async job submit for node 69ae73cf8ef4424e487b61dc 2026-03-11 07:16:37,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:16:37,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73cf8ef4424e487b61dc HTTP/1.1" 200 None 2026-03-11 07:16:37,238 - lava_callback - DEBUG - Processing node 69ae73cf8ef4424e487b61dc: baseline-arm-clabbe 2026-03-11 07:16:37,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae73cf8ef4424e487b61dc 2026-03-11 07:16:37,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:16:37,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73cf8ef4424e487b61dc HTTP/1.1" 200 None 2026-03-11 07:16:37,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:16:37,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 07:16:37,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:16:37,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae73cf8ef4424e487b61dc HTTP/1.1" 200 None 2026-03-11 07:16:37,339 - lava_callback - ERROR - Error processing callback for node 69ae73cf8ef4424e487b61dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42118 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:17:17,191 - lava_callback - INFO - Received callback for node 69ae73f88ef4424e487b6201 INFO: 172.21.0.1:42132 - "POST /node/69ae73f88ef4424e487b6201 HTTP/1.0" 202 Accepted 2026-03-11 07:17:17,196 - lava_callback - INFO - Starting async job submit for node 69ae73f88ef4424e487b6201 2026-03-11 07:17:17,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:17:17,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73f88ef4424e487b6201 HTTP/1.1" 200 None 2026-03-11 07:17:17,213 - lava_callback - DEBUG - Processing node 69ae73f88ef4424e487b6201: baseline-arm-clabbe 2026-03-11 07:17:17,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae73f88ef4424e487b6201 2026-03-11 07:17:17,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:17:17,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae73f88ef4424e487b6201 HTTP/1.1" 200 None 2026-03-11 07:17:17,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:17:17,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 07:17:17,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:17:17,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae73f88ef4424e487b6201 HTTP/1.1" 200 None 2026-03-11 07:17:17,275 - lava_callback - ERROR - Error processing callback for node 69ae73f88ef4424e487b6201: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54100 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:18:17,266 - lava_callback - INFO - Received callback for node 69ae742e8ef4424e487b6251 INFO: 172.21.0.1:54102 - "POST /node/69ae742e8ef4424e487b6251 HTTP/1.0" 202 Accepted 2026-03-11 07:18:17,272 - lava_callback - INFO - Starting async job submit for node 69ae742e8ef4424e487b6251 2026-03-11 07:18:17,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:18:17,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae742e8ef4424e487b6251 HTTP/1.1" 200 None 2026-03-11 07:18:17,288 - lava_callback - DEBUG - Processing node 69ae742e8ef4424e487b6251: baseline-arm64-clabbe 2026-03-11 07:18:17,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae742e8ef4424e487b6251 2026-03-11 07:18:17,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:18:17,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae742e8ef4424e487b6251 HTTP/1.1" 200 None 2026-03-11 07:18:17,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:18:17,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbcf911a9cbe7f77ccbd HTTP/1.1" 200 None 2026-03-11 07:18:17,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:18:17,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae742e8ef4424e487b6251 HTTP/1.1" 200 None 2026-03-11 07:18:17,381 - lava_callback - ERROR - Error processing callback for node 69ae742e8ef4424e487b6251: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33824 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:24:31,032 - lava_callback - INFO - Received callback for node 69ae75a08ef4424e487b62bd INFO: 172.21.0.1:33828 - "POST /node/69ae75a08ef4424e487b62bd HTTP/1.0" 202 Accepted 2026-03-11 07:24:31,038 - lava_callback - INFO - Starting async job submit for node 69ae75a08ef4424e487b62bd 2026-03-11 07:24:31,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:24:31,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae75a08ef4424e487b62bd HTTP/1.1" 200 None 2026-03-11 07:24:31,055 - lava_callback - DEBUG - Processing node 69ae75a08ef4424e487b62bd: baseline-arm64-baylibre 2026-03-11 07:24:31,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae75a08ef4424e487b62bd 2026-03-11 07:24:31,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:24:31,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae75a08ef4424e487b62bd HTTP/1.1" 200 None 2026-03-11 07:24:31,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:24:31,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:24:31,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:24:31,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae75a08ef4424e487b62bd HTTP/1.1" 200 None 2026-03-11 07:24:31,128 - lava_callback - ERROR - Error processing callback for node 69ae75a08ef4424e487b62bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49628 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:25:51,127 - lava_callback - INFO - Received callback for node 69ae75ec8ef4424e487b62d5 INFO: 172.21.0.1:40052 - "POST /node/69ae75ec8ef4424e487b62d5 HTTP/1.0" 202 Accepted 2026-03-11 07:25:51,133 - lava_callback - INFO - Starting async job submit for node 69ae75ec8ef4424e487b62d5 2026-03-11 07:25:51,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:25:51,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae75ec8ef4424e487b62d5 HTTP/1.1" 200 None 2026-03-11 07:25:51,148 - lava_callback - DEBUG - Processing node 69ae75ec8ef4424e487b62d5: baseline-arm64-baylibre 2026-03-11 07:25:51,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae75ec8ef4424e487b62d5 2026-03-11 07:25:51,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:25:51,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae75ec8ef4424e487b62d5 HTTP/1.1" 200 None 2026-03-11 07:25:51,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:25:51,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:25:51,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:25:51,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae75ec8ef4424e487b62d5 HTTP/1.1" 200 None 2026-03-11 07:25:51,236 - lava_callback - ERROR - Error processing callback for node 69ae75ec8ef4424e487b62d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49426 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:27:31,152 - lava_callback - INFO - Received callback for node 69ae764d8ef4424e487b62e9 INFO: 172.21.0.1:49438 - "POST /node/69ae764d8ef4424e487b62e9 HTTP/1.0" 202 Accepted 2026-03-11 07:27:31,158 - lava_callback - INFO - Starting async job submit for node 69ae764d8ef4424e487b62e9 2026-03-11 07:27:31,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:27:31,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae764d8ef4424e487b62e9 HTTP/1.1" 200 None 2026-03-11 07:27:31,178 - lava_callback - DEBUG - Processing node 69ae764d8ef4424e487b62e9: baseline-arm64-baylibre 2026-03-11 07:27:31,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae764d8ef4424e487b62e9 2026-03-11 07:27:31,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:27:31,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae764d8ef4424e487b62e9 HTTP/1.1" 200 None 2026-03-11 07:27:31,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:27:31,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:27:31,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:27:31,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae764d8ef4424e487b62e9 HTTP/1.1" 200 None 2026-03-11 07:27:31,234 - lava_callback - ERROR - Error processing callback for node 69ae764d8ef4424e487b62e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36680 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:30:11,579 - lava_callback - INFO - Received callback for node 69ae76f28ef4424e487b6322 INFO: 172.21.0.1:43514 - "POST /node/69ae76f28ef4424e487b6322 HTTP/1.0" 202 Accepted 2026-03-11 07:30:11,586 - lava_callback - INFO - Starting async job submit for node 69ae76f28ef4424e487b6322 2026-03-11 07:30:11,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:30:11,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae76f28ef4424e487b6322 HTTP/1.1" 200 None 2026-03-11 07:30:11,601 - lava_callback - DEBUG - Processing node 69ae76f28ef4424e487b6322: baseline-arm64-baylibre 2026-03-11 07:30:11,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae76f28ef4424e487b6322 2026-03-11 07:30:11,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:30:11,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae76f28ef4424e487b6322 HTTP/1.1" 200 None 2026-03-11 07:30:11,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:30:11,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:30:11,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:30:11,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae76f28ef4424e487b6322 HTTP/1.1" 200 None 2026-03-11 07:30:11,653 - lava_callback - ERROR - Error processing callback for node 69ae76f28ef4424e487b6322: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53638 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:31:31,463 - lava_callback - INFO - Received callback for node 69ae773f8ef4424e487b6339 INFO: 172.21.0.1:53642 - "POST /node/69ae773f8ef4424e487b6339 HTTP/1.0" 202 Accepted 2026-03-11 07:31:31,469 - lava_callback - INFO - Starting async job submit for node 69ae773f8ef4424e487b6339 2026-03-11 07:31:31,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:31:31,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae773f8ef4424e487b6339 HTTP/1.1" 200 None 2026-03-11 07:31:31,486 - lava_callback - DEBUG - Processing node 69ae773f8ef4424e487b6339: baseline-arm64-baylibre 2026-03-11 07:31:31,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae773f8ef4424e487b6339 2026-03-11 07:31:31,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:31:31,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae773f8ef4424e487b6339 HTTP/1.1" 200 None 2026-03-11 07:31:31,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:31:31,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:31:31,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:31:31,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae773f8ef4424e487b6339 HTTP/1.1" 200 None 2026-03-11 07:31:31,565 - lava_callback - ERROR - Error processing callback for node 69ae773f8ef4424e487b6339: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:33:57,989 - lava_callback - INFO - Received callback for node 69ae77de8ef4424e487b6374 INFO: 172.21.0.1:44936 - "POST /node/69ae77de8ef4424e487b6374 HTTP/1.0" 202 Accepted 2026-03-11 07:33:57,995 - lava_callback - INFO - Starting async job submit for node 69ae77de8ef4424e487b6374 2026-03-11 07:33:57,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:33:58,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae77de8ef4424e487b6374 HTTP/1.1" 200 None 2026-03-11 07:33:58,012 - lava_callback - DEBUG - Processing node 69ae77de8ef4424e487b6374: baseline-arm64-clabbe 2026-03-11 07:33:58,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae77de8ef4424e487b6374 2026-03-11 07:33:58,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:33:58,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae77de8ef4424e487b6374 HTTP/1.1" 200 None 2026-03-11 07:33:58,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:33:58,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:33:58,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:33:58,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae77de8ef4424e487b6374 HTTP/1.1" 200 None 2026-03-11 07:33:58,069 - lava_callback - ERROR - Error processing callback for node 69ae77de8ef4424e487b6374: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46510 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:35:18,033 - lava_callback - INFO - Received callback for node 69ae782a8ef4424e487b6389 INFO: 172.21.0.1:46516 - "POST /node/69ae782a8ef4424e487b6389 HTTP/1.0" 202 Accepted 2026-03-11 07:35:18,039 - lava_callback - INFO - Starting async job submit for node 69ae782a8ef4424e487b6389 2026-03-11 07:35:18,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:35:18,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae782a8ef4424e487b6389 HTTP/1.1" 200 None 2026-03-11 07:35:18,054 - lava_callback - DEBUG - Processing node 69ae782a8ef4424e487b6389: baseline-arm64-clabbe 2026-03-11 07:35:18,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae782a8ef4424e487b6389 2026-03-11 07:35:18,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:35:18,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae782a8ef4424e487b6389 HTTP/1.1" 200 None 2026-03-11 07:35:18,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:35:18,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:35:18,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:35:18,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae782a8ef4424e487b6389 HTTP/1.1" 200 None 2026-03-11 07:35:18,129 - lava_callback - ERROR - Error processing callback for node 69ae782a8ef4424e487b6389: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36628 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:36:38,071 - lava_callback - INFO - Received callback for node 69ae78778ef4424e487b63ab INFO: 172.21.0.1:36634 - "POST /node/69ae78778ef4424e487b63ab HTTP/1.0" 202 Accepted 2026-03-11 07:36:38,077 - lava_callback - INFO - Starting async job submit for node 69ae78778ef4424e487b63ab 2026-03-11 07:36:38,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:36:38,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae78778ef4424e487b63ab HTTP/1.1" 200 None 2026-03-11 07:36:38,092 - lava_callback - DEBUG - Processing node 69ae78778ef4424e487b63ab: baseline-arm64-clabbe 2026-03-11 07:36:38,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae78778ef4424e487b63ab 2026-03-11 07:36:38,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:36:38,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae78778ef4424e487b63ab HTTP/1.1" 200 None 2026-03-11 07:36:38,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:36:38,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:36:38,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:36:38,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae78778ef4424e487b63ab HTTP/1.1" 200 None 2026-03-11 07:36:38,153 - lava_callback - ERROR - Error processing callback for node 69ae78778ef4424e487b63ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35510 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:37:58,200 - lava_callback - INFO - Received callback for node 69ae78c58ef4424e487b63b7 INFO: 172.21.0.1:35522 - "POST /node/69ae78c58ef4424e487b63b7 HTTP/1.0" 202 Accepted 2026-03-11 07:37:58,206 - lava_callback - INFO - Starting async job submit for node 69ae78c58ef4424e487b63b7 2026-03-11 07:37:58,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:37:58,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae78c58ef4424e487b63b7 HTTP/1.1" 200 None 2026-03-11 07:37:58,221 - lava_callback - DEBUG - Processing node 69ae78c58ef4424e487b63b7: baseline-arm64-clabbe 2026-03-11 07:37:58,222 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae78c58ef4424e487b63b7 2026-03-11 07:37:58,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:37:58,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae78c58ef4424e487b63b7 HTTP/1.1" 200 None 2026-03-11 07:37:58,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:37:58,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:37:58,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:37:58,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae78c58ef4424e487b63b7 HTTP/1.1" 200 None 2026-03-11 07:37:58,297 - lava_callback - ERROR - Error processing callback for node 69ae78c58ef4424e487b63b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34074 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:39:18,193 - lava_callback - INFO - Received callback for node 69ae79148ef4424e487b63da INFO: 172.21.0.1:34090 - "POST /node/69ae79148ef4424e487b63da HTTP/1.0" 202 Accepted 2026-03-11 07:39:18,199 - lava_callback - INFO - Starting async job submit for node 69ae79148ef4424e487b63da 2026-03-11 07:39:18,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:39:18,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79148ef4424e487b63da HTTP/1.1" 200 None 2026-03-11 07:39:18,215 - lava_callback - DEBUG - Processing node 69ae79148ef4424e487b63da: baseline-arm64-clabbe 2026-03-11 07:39:18,215 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae79148ef4424e487b63da 2026-03-11 07:39:18,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:39:18,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79148ef4424e487b63da HTTP/1.1" 200 None 2026-03-11 07:39:18,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:39:18,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:39:18,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:39:18,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae79148ef4424e487b63da HTTP/1.1" 200 None 2026-03-11 07:39:18,280 - lava_callback - ERROR - Error processing callback for node 69ae79148ef4424e487b63da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35082 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:40:38,218 - lava_callback - INFO - Received callback for node 69ae79628ef4424e487b63eb INFO: 172.21.0.1:35086 - "POST /node/69ae79628ef4424e487b63eb HTTP/1.0" 202 Accepted 2026-03-11 07:40:38,224 - lava_callback - INFO - Starting async job submit for node 69ae79628ef4424e487b63eb 2026-03-11 07:40:38,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:40:38,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79628ef4424e487b63eb HTTP/1.1" 200 None 2026-03-11 07:40:38,240 - lava_callback - DEBUG - Processing node 69ae79628ef4424e487b63eb: baseline-arm64-clabbe 2026-03-11 07:40:38,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae79628ef4424e487b63eb 2026-03-11 07:40:38,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:40:38,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79628ef4424e487b63eb HTTP/1.1" 200 None 2026-03-11 07:40:38,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:40:38,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:40:38,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:40:38,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae79628ef4424e487b63eb HTTP/1.1" 200 None 2026-03-11 07:40:38,305 - lava_callback - ERROR - Error processing callback for node 69ae79628ef4424e487b63eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39306 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:41:38,245 - lava_callback - INFO - Received callback for node 69ae79a28ef4424e487b6406 INFO: 172.21.0.1:39318 - "POST /node/69ae79a28ef4424e487b6406 HTTP/1.0" 202 Accepted 2026-03-11 07:41:38,251 - lava_callback - INFO - Starting async job submit for node 69ae79a28ef4424e487b6406 2026-03-11 07:41:38,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:41:38,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79a28ef4424e487b6406 HTTP/1.1" 200 None 2026-03-11 07:41:38,269 - lava_callback - DEBUG - Processing node 69ae79a28ef4424e487b6406: baseline-arm-clabbe 2026-03-11 07:41:38,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae79a28ef4424e487b6406 2026-03-11 07:41:38,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:41:38,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79a28ef4424e487b6406 HTTP/1.1" 200 None 2026-03-11 07:41:38,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:41:38,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 07:41:38,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:41:38,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae79a28ef4424e487b6406 HTTP/1.1" 200 None 2026-03-11 07:41:38,327 - lava_callback - ERROR - Error processing callback for node 69ae79a28ef4424e487b6406: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:42:18,245 - lava_callback - INFO - Received callback for node 69ae79cb8ef4424e487b6426 INFO: 172.21.0.1:35912 - "POST /node/69ae79cb8ef4424e487b6426 HTTP/1.0" 202 Accepted 2026-03-11 07:42:18,251 - lava_callback - INFO - Starting async job submit for node 69ae79cb8ef4424e487b6426 2026-03-11 07:42:18,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:18,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79cb8ef4424e487b6426 HTTP/1.1" 200 None 2026-03-11 07:42:18,267 - lava_callback - DEBUG - Processing node 69ae79cb8ef4424e487b6426: baseline-arm-clabbe 2026-03-11 07:42:18,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae79cb8ef4424e487b6426 2026-03-11 07:42:18,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:18,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79cb8ef4424e487b6426 HTTP/1.1" 200 None 2026-03-11 07:42:18,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:18,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a8cbbb911a9cbe7f77ccab HTTP/1.1" 200 None 2026-03-11 07:42:18,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:18,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae79cb8ef4424e487b6426 HTTP/1.1" 200 None 2026-03-11 07:42:18,330 - lava_callback - ERROR - Error processing callback for node 69ae79cb8ef4424e487b6426: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:42:52,110 - lava_callback - INFO - Received callback for node 69ae79f78ef4424e487b6470 INFO: 172.21.0.1:57256 - "POST /node/69ae79f78ef4424e487b6470 HTTP/1.0" 202 Accepted 2026-03-11 07:42:52,115 - lava_callback - INFO - Starting async job submit for node 69ae79f78ef4424e487b6470 2026-03-11 07:42:52,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:52,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79f78ef4424e487b6470 HTTP/1.1" 200 None 2026-03-11 07:42:52,131 - lava_callback - DEBUG - Processing node 69ae79f78ef4424e487b6470: baseline-arm64-baylibre 2026-03-11 07:42:52,131 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae79f78ef4424e487b6470 2026-03-11 07:42:52,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:52,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae79f78ef4424e487b6470 HTTP/1.1" 200 None 2026-03-11 07:42:52,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:52,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:42:52,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:42:52,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae79f78ef4424e487b6470 HTTP/1.1" 200 None 2026-03-11 07:42:52,194 - lava_callback - ERROR - Error processing callback for node 69ae79f78ef4424e487b6470: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54118 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:44:12,183 - lava_callback - INFO - Received callback for node 69ae7a458ef4424e487b648f INFO: 172.21.0.1:37078 - "POST /node/69ae7a458ef4424e487b648f HTTP/1.0" 202 Accepted 2026-03-11 07:44:12,188 - lava_callback - INFO - Starting async job submit for node 69ae7a458ef4424e487b648f 2026-03-11 07:44:12,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:44:12,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7a458ef4424e487b648f HTTP/1.1" 200 None 2026-03-11 07:44:12,204 - lava_callback - DEBUG - Processing node 69ae7a458ef4424e487b648f: baseline-arm64-baylibre 2026-03-11 07:44:12,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7a458ef4424e487b648f 2026-03-11 07:44:12,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:44:12,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7a458ef4424e487b648f HTTP/1.1" 200 None 2026-03-11 07:44:12,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:44:12,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:44:12,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:44:12,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7a458ef4424e487b648f HTTP/1.1" 200 None 2026-03-11 07:44:12,261 - lava_callback - ERROR - Error processing callback for node 69ae7a458ef4424e487b648f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36352 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:45:32,287 - lava_callback - INFO - Received callback for node 69ae7a928ef4424e487b64ad INFO: 172.21.0.1:36366 - "POST /node/69ae7a928ef4424e487b64ad HTTP/1.0" 202 Accepted 2026-03-11 07:45:32,294 - lava_callback - INFO - Starting async job submit for node 69ae7a928ef4424e487b64ad 2026-03-11 07:45:32,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:45:32,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7a928ef4424e487b64ad HTTP/1.1" 200 None 2026-03-11 07:45:32,312 - lava_callback - DEBUG - Processing node 69ae7a928ef4424e487b64ad: baseline-arm64-baylibre 2026-03-11 07:45:32,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7a928ef4424e487b64ad 2026-03-11 07:45:32,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:45:32,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7a928ef4424e487b64ad HTTP/1.1" 200 None 2026-03-11 07:45:32,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:45:32,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:45:32,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:45:32,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7a928ef4424e487b64ad HTTP/1.1" 200 None 2026-03-11 07:45:32,403 - lava_callback - ERROR - Error processing callback for node 69ae7a928ef4424e487b64ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55544 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:46:52,324 - lava_callback - INFO - Received callback for node 69ae7add8ef4424e487b64bb INFO: 172.21.0.1:48536 - "POST /node/69ae7add8ef4424e487b64bb HTTP/1.0" 202 Accepted 2026-03-11 07:46:52,330 - lava_callback - INFO - Starting async job submit for node 69ae7add8ef4424e487b64bb 2026-03-11 07:46:52,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:46:52,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7add8ef4424e487b64bb HTTP/1.1" 200 None 2026-03-11 07:46:52,350 - lava_callback - DEBUG - Processing node 69ae7add8ef4424e487b64bb: baseline-arm64-baylibre 2026-03-11 07:46:52,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7add8ef4424e487b64bb 2026-03-11 07:46:52,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:46:52,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7add8ef4424e487b64bb HTTP/1.1" 200 None 2026-03-11 07:46:52,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:46:52,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:46:52,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:46:52,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7add8ef4424e487b64bb HTTP/1.1" 200 None 2026-03-11 07:46:52,402 - lava_callback - ERROR - Error processing callback for node 69ae7add8ef4424e487b64bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:48:12,755 - lava_callback - INFO - Received callback for node 69ae7b288ef4424e487b64db INFO: 172.21.0.1:40990 - "POST /node/69ae7b288ef4424e487b64db HTTP/1.0" 202 Accepted 2026-03-11 07:48:12,760 - lava_callback - INFO - Starting async job submit for node 69ae7b288ef4424e487b64db 2026-03-11 07:48:12,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:48:12,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7b288ef4424e487b64db HTTP/1.1" 200 None 2026-03-11 07:48:12,779 - lava_callback - DEBUG - Processing node 69ae7b288ef4424e487b64db: baseline-arm64-baylibre 2026-03-11 07:48:12,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7b288ef4424e487b64db 2026-03-11 07:48:12,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:48:12,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7b288ef4424e487b64db HTTP/1.1" 200 None 2026-03-11 07:48:12,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:48:12,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:48:12,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:48:12,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7b288ef4424e487b64db HTTP/1.1" 200 None 2026-03-11 07:48:12,839 - lava_callback - ERROR - Error processing callback for node 69ae7b288ef4424e487b64db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45566 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:49:12,419 - lava_callback - INFO - Received callback for node 69ae7b748ef4424e487b64f1 INFO: 172.21.0.1:51758 - "POST /node/69ae7b748ef4424e487b64f1 HTTP/1.0" 202 Accepted 2026-03-11 07:49:12,424 - lava_callback - INFO - Starting async job submit for node 69ae7b748ef4424e487b64f1 2026-03-11 07:49:12,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:49:12,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7b748ef4424e487b64f1 HTTP/1.1" 200 None 2026-03-11 07:49:12,441 - lava_callback - DEBUG - Processing node 69ae7b748ef4424e487b64f1: baseline-arm64-baylibre 2026-03-11 07:49:12,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7b748ef4424e487b64f1 2026-03-11 07:49:12,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:49:12,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7b748ef4424e487b64f1 HTTP/1.1" 200 None 2026-03-11 07:49:12,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:49:12,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:49:12,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:49:12,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7b748ef4424e487b64f1 HTTP/1.1" 200 None 2026-03-11 07:49:12,497 - lava_callback - ERROR - Error processing callback for node 69ae7b748ef4424e487b64f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47496 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:50:32,455 - lava_callback - INFO - Received callback for node 69ae7bc28ef4424e487b6511 INFO: 172.21.0.1:47510 - "POST /node/69ae7bc28ef4424e487b6511 HTTP/1.0" 202 Accepted 2026-03-11 07:50:32,461 - lava_callback - INFO - Starting async job submit for node 69ae7bc28ef4424e487b6511 2026-03-11 07:50:32,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:50:32,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7bc28ef4424e487b6511 HTTP/1.1" 200 None 2026-03-11 07:50:32,476 - lava_callback - DEBUG - Processing node 69ae7bc28ef4424e487b6511: baseline-arm64-baylibre 2026-03-11 07:50:32,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7bc28ef4424e487b6511 2026-03-11 07:50:32,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:50:32,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7bc28ef4424e487b6511 HTTP/1.1" 200 None 2026-03-11 07:50:32,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:50:32,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae11758ef4424e487b2eb1 HTTP/1.1" 200 None 2026-03-11 07:50:32,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:50:32,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7bc28ef4424e487b6511 HTTP/1.1" 200 None 2026-03-11 07:50:32,545 - lava_callback - ERROR - Error processing callback for node 69ae7bc28ef4424e487b6511: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42694 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:51:52,570 - lava_callback - INFO - Received callback for node 69ae7c0e8ef4424e487b6530 INFO: 172.21.0.1:44828 - "POST /node/69ae7c0e8ef4424e487b6530 HTTP/1.0" 202 Accepted 2026-03-11 07:51:52,576 - lava_callback - INFO - Starting async job submit for node 69ae7c0e8ef4424e487b6530 2026-03-11 07:51:52,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:51:52,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7c0e8ef4424e487b6530 HTTP/1.1" 200 None 2026-03-11 07:51:52,592 - lava_callback - DEBUG - Processing node 69ae7c0e8ef4424e487b6530: baseline-arm64-baylibre 2026-03-11 07:51:52,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7c0e8ef4424e487b6530 2026-03-11 07:51:52,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:51:52,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7c0e8ef4424e487b6530 HTTP/1.1" 200 None 2026-03-11 07:51:52,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:51:52,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:51:52,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:51:52,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7c0e8ef4424e487b6530 HTTP/1.1" 200 None 2026-03-11 07:51:52,645 - lava_callback - ERROR - Error processing callback for node 69ae7c0e8ef4424e487b6530: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42650 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:53:12,609 - lava_callback - INFO - Received callback for node 69ae7c598ef4424e487b6543 INFO: 172.21.0.1:47750 - "POST /node/69ae7c598ef4424e487b6543 HTTP/1.0" 202 Accepted 2026-03-11 07:53:12,614 - lava_callback - INFO - Starting async job submit for node 69ae7c598ef4424e487b6543 2026-03-11 07:53:12,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:53:12,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7c598ef4424e487b6543 HTTP/1.1" 200 None 2026-03-11 07:53:12,634 - lava_callback - DEBUG - Processing node 69ae7c598ef4424e487b6543: baseline-arm64-baylibre 2026-03-11 07:53:12,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7c598ef4424e487b6543 2026-03-11 07:53:12,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:53:12,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7c598ef4424e487b6543 HTTP/1.1" 200 None 2026-03-11 07:53:12,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:53:12,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:53:12,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:53:12,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7c598ef4424e487b6543 HTTP/1.1" 200 None 2026-03-11 07:53:12,685 - lava_callback - ERROR - Error processing callback for node 69ae7c598ef4424e487b6543: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43022 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:54:32,659 - lava_callback - INFO - Received callback for node 69ae7ca68ef4424e487b6559 INFO: 172.21.0.1:43024 - "POST /node/69ae7ca68ef4424e487b6559 HTTP/1.0" 202 Accepted 2026-03-11 07:54:32,665 - lava_callback - INFO - Starting async job submit for node 69ae7ca68ef4424e487b6559 2026-03-11 07:54:32,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:54:32,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7ca68ef4424e487b6559 HTTP/1.1" 200 None 2026-03-11 07:54:32,681 - lava_callback - DEBUG - Processing node 69ae7ca68ef4424e487b6559: baseline-arm64-baylibre 2026-03-11 07:54:32,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7ca68ef4424e487b6559 2026-03-11 07:54:32,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:54:32,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7ca68ef4424e487b6559 HTTP/1.1" 200 None 2026-03-11 07:54:32,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:54:32,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:54:32,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:54:32,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7ca68ef4424e487b6559 HTTP/1.1" 200 None 2026-03-11 07:54:32,745 - lava_callback - ERROR - Error processing callback for node 69ae7ca68ef4424e487b6559: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58868 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:55:32,720 - lava_callback - INFO - Received callback for node 69ae7cf18ef4424e487b656a INFO: 172.21.0.1:58876 - "POST /node/69ae7cf18ef4424e487b656a HTTP/1.0" 202 Accepted 2026-03-11 07:55:32,726 - lava_callback - INFO - Starting async job submit for node 69ae7cf18ef4424e487b656a 2026-03-11 07:55:32,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:55:32,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7cf18ef4424e487b656a HTTP/1.1" 200 None 2026-03-11 07:55:32,743 - lava_callback - DEBUG - Processing node 69ae7cf18ef4424e487b656a: baseline-arm64-baylibre 2026-03-11 07:55:32,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7cf18ef4424e487b656a 2026-03-11 07:55:32,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:55:32,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7cf18ef4424e487b656a HTTP/1.1" 200 None 2026-03-11 07:55:32,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:55:32,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:55:32,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:55:32,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7cf18ef4424e487b656a HTTP/1.1" 200 None 2026-03-11 07:55:32,802 - lava_callback - ERROR - Error processing callback for node 69ae7cf18ef4424e487b656a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34010 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:56:52,868 - lava_callback - INFO - Received callback for node 69ae7d3d8ef4424e487b6580 INFO: 172.21.0.1:36374 - "POST /node/69ae7d3d8ef4424e487b6580 HTTP/1.0" 202 Accepted 2026-03-11 07:56:52,873 - lava_callback - INFO - Starting async job submit for node 69ae7d3d8ef4424e487b6580 2026-03-11 07:56:52,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:56:52,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7d3d8ef4424e487b6580 HTTP/1.1" 200 None 2026-03-11 07:56:52,891 - lava_callback - DEBUG - Processing node 69ae7d3d8ef4424e487b6580: baseline-arm64-baylibre 2026-03-11 07:56:52,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7d3d8ef4424e487b6580 2026-03-11 07:56:52,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:56:52,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7d3d8ef4424e487b6580 HTTP/1.1" 200 None 2026-03-11 07:56:52,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:56:52,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:56:52,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:56:52,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7d3d8ef4424e487b6580 HTTP/1.1" 200 None 2026-03-11 07:56:52,943 - lava_callback - ERROR - Error processing callback for node 69ae7d3d8ef4424e487b6580: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:58:12,847 - lava_callback - INFO - Received callback for node 69ae7d8f8ef4424e487b659c INFO: 172.21.0.1:49650 - "POST /node/69ae7d8f8ef4424e487b659c HTTP/1.0" 202 Accepted 2026-03-11 07:58:12,852 - lava_callback - INFO - Starting async job submit for node 69ae7d8f8ef4424e487b659c 2026-03-11 07:58:12,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:58:12,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7d8f8ef4424e487b659c HTTP/1.1" 200 None 2026-03-11 07:58:12,869 - lava_callback - DEBUG - Processing node 69ae7d8f8ef4424e487b659c: baseline-arm64-baylibre 2026-03-11 07:58:12,869 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7d8f8ef4424e487b659c 2026-03-11 07:58:12,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:58:12,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7d8f8ef4424e487b659c HTTP/1.1" 200 None 2026-03-11 07:58:12,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:58:12,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:58:12,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:58:12,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7d8f8ef4424e487b659c HTTP/1.1" 200 None 2026-03-11 07:58:12,951 - lava_callback - ERROR - Error processing callback for node 69ae7d8f8ef4424e487b659c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46888 - "GET / HTTP/1.0" 200 OK 2026-03-11 07:59:39,000 - lava_callback - INFO - Received callback for node 69ae7de18ef4424e487b65b6 INFO: 172.21.0.1:46898 - "POST /node/69ae7de18ef4424e487b65b6 HTTP/1.0" 202 Accepted 2026-03-11 07:59:39,006 - lava_callback - INFO - Starting async job submit for node 69ae7de18ef4424e487b65b6 2026-03-11 07:59:39,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:59:39,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7de18ef4424e487b65b6 HTTP/1.1" 200 None 2026-03-11 07:59:39,023 - lava_callback - DEBUG - Processing node 69ae7de18ef4424e487b65b6: baseline-arm64-clabbe 2026-03-11 07:59:39,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7de18ef4424e487b65b6 2026-03-11 07:59:39,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:59:39,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7de18ef4424e487b65b6 HTTP/1.1" 200 None 2026-03-11 07:59:39,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:59:39,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 07:59:39,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 07:59:39,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7de18ef4424e487b65b6 HTTP/1.1" 200 None 2026-03-11 07:59:39,083 - lava_callback - ERROR - Error processing callback for node 69ae7de18ef4424e487b65b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52200 - "GET / HTTP/1.0" 200 OK 2026-03-11 08:00:58,985 - lava_callback - INFO - Received callback for node 69ae7e2e8ef4424e487b65cb INFO: 172.21.0.1:52208 - "POST /node/69ae7e2e8ef4424e487b65cb HTTP/1.0" 202 Accepted 2026-03-11 08:00:58,991 - lava_callback - INFO - Starting async job submit for node 69ae7e2e8ef4424e487b65cb 2026-03-11 08:00:58,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:00:59,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7e2e8ef4424e487b65cb HTTP/1.1" 200 None 2026-03-11 08:00:59,009 - lava_callback - DEBUG - Processing node 69ae7e2e8ef4424e487b65cb: baseline-arm64-clabbe 2026-03-11 08:00:59,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7e2e8ef4424e487b65cb 2026-03-11 08:00:59,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:00:59,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7e2e8ef4424e487b65cb HTTP/1.1" 200 None 2026-03-11 08:00:59,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:00:59,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 08:00:59,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:00:59,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7e2e8ef4424e487b65cb HTTP/1.1" 200 None 2026-03-11 08:00:59,068 - lava_callback - ERROR - Error processing callback for node 69ae7e2e8ef4424e487b65cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43278 - "GET / HTTP/1.0" 200 OK 2026-03-11 08:02:19,013 - lava_callback - INFO - Received callback for node 69ae7e7a8ef4424e487b65e5 INFO: 172.21.0.1:43286 - "POST /node/69ae7e7a8ef4424e487b65e5 HTTP/1.0" 202 Accepted 2026-03-11 08:02:19,019 - lava_callback - INFO - Starting async job submit for node 69ae7e7a8ef4424e487b65e5 2026-03-11 08:02:19,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:02:19,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7e7a8ef4424e487b65e5 HTTP/1.1" 200 None 2026-03-11 08:02:19,040 - lava_callback - DEBUG - Processing node 69ae7e7a8ef4424e487b65e5: baseline-arm64-clabbe 2026-03-11 08:02:19,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7e7a8ef4424e487b65e5 2026-03-11 08:02:19,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:02:19,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7e7a8ef4424e487b65e5 HTTP/1.1" 200 None 2026-03-11 08:02:19,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:02:19,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 08:02:19,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:02:19,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7e7a8ef4424e487b65e5 HTTP/1.1" 200 None 2026-03-11 08:02:19,107 - lava_callback - ERROR - Error processing callback for node 69ae7e7a8ef4424e487b65e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41252 - "GET / HTTP/1.0" 200 OK 2026-03-11 08:03:39,093 - lava_callback - INFO - Received callback for node 69ae7ec88ef4424e487b65f9 INFO: 172.21.0.1:41258 - "POST /node/69ae7ec88ef4424e487b65f9 HTTP/1.0" 202 Accepted 2026-03-11 08:03:39,099 - lava_callback - INFO - Starting async job submit for node 69ae7ec88ef4424e487b65f9 2026-03-11 08:03:39,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:03:39,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7ec88ef4424e487b65f9 HTTP/1.1" 200 None 2026-03-11 08:03:39,125 - lava_callback - DEBUG - Processing node 69ae7ec88ef4424e487b65f9: baseline-arm64-clabbe 2026-03-11 08:03:39,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ae7ec88ef4424e487b65f9 2026-03-11 08:03:39,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:03:39,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae7ec88ef4424e487b65f9 HTTP/1.1" 200 None 2026-03-11 08:03:39,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:03:39,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-11 08:03:39,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-11 08:03:39,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae7ec88ef4424e487b65f9 HTTP/1.1" 200 None 2026-03-11 08:03:39,178 - lava_callback - ERROR - Error processing callback for node 69ae7ec88ef4424e487b65f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]