INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:28:45,505 - lava_callback - INFO - Received callback for node 69a1be5669d5b6151c03ae23 2026-02-27 16:28:45,554 - lava_callback - INFO - Starting async job submit for node 69a1be5669d5b6151c03ae23 INFO: 172.21.0.1:47822 - "POST /node/69a1be5669d5b6151c03ae23 HTTP/1.0" 202 Accepted 2026-02-27 16:28:45,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:45,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1be5669d5b6151c03ae23 HTTP/1.1" 200 None 2026-02-27 16:28:45,600 - lava_callback - DEBUG - Processing node 69a1be5669d5b6151c03ae23: baseline-arm-pengutronix 2026-02-27 16:28:45,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:28:45,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:28:45,860 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69a1be5669d5b6151c03ae23/log.txt.gz 2026-02-27 16:28:45,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:28:46,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:28:46,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69a1be5669d5b6151c03ae23/lava-logs.yaml 2026-02-27 16:28:46,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:28:46,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:28:46,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69a1be5669d5b6151c03ae23/lava_callback.json.gz DEBUG: Setup failed for 69a1be5669d5b6151c03ae23. Transitting job node result: incomplete -> fail 2026-02-27 16:28:46,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:46,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1be5669d5b6151c03ae23 HTTP/1.1" 200 None 2026-02-27 16:28:46,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:46,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1b211d15a97544f0cb9d7 HTTP/1.1" 200 None 2026-02-27 16:28:46,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:46,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69a1be5669d5b6151c03ae23 HTTP/1.1" 200 None 2026-02-27 16:28:46,410 - lava_callback - INFO - Completed processing callback for node 69a1be5669d5b6151c03ae23 INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34076 - "GET /api/metrics HTTP/1.0" 200 OK 2026-02-27 16:29:16,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:16,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33932 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:29:37,141 - lava_callback - INFO - Received callback for node 69a1c27c5536d4cbaf00ef64 INFO: 172.21.0.1:33934 - "POST /node/69a1c27c5536d4cbaf00ef64 HTTP/1.0" 202 Accepted 2026-02-27 16:29:37,148 - lava_callback - INFO - Starting async job submit for node 69a1c27c5536d4cbaf00ef64 2026-02-27 16:29:37,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:37,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c27c5536d4cbaf00ef64 HTTP/1.1" 200 None 2026-02-27 16:29:37,220 - lava_callback - DEBUG - Processing node 69a1c27c5536d4cbaf00ef64: baseline-arm 2026-02-27 16:29:37,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:29:37,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:29:37,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69a1c27c5536d4cbaf00ef64/log.txt.gz 2026-02-27 16:29:38,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:29:38,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:29:38,584 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69a1c27c5536d4cbaf00ef64/lava-logs.yaml 2026-02-27 16:29:38,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:29:38,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:29:38,792 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69a1c27c5536d4cbaf00ef64/lava_callback.json.gz DEBUG: Setup failed for 69a1c27c5536d4cbaf00ef64. Transitting job node result: incomplete -> fail 2026-02-27 16:29:38,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:38,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c27c5536d4cbaf00ef64 HTTP/1.1" 200 None 2026-02-27 16:29:38,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:38,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1b211d15a97544f0cb9d7 HTTP/1.1" 200 None 2026-02-27 16:29:38,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:38,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69a1c27c5536d4cbaf00ef64 HTTP/1.1" 200 None 2026-02-27 16:29:39,069 - lava_callback - INFO - Completed processing callback for node 69a1c27c5536d4cbaf00ef64 2026-02-27 16:29:46,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:46,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39008 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:40:11,400 - lava_callback - INFO - Received callback for node 69a1c4615536d4cbaf00efcc INFO: 172.21.0.1:44146 - "POST /node/69a1c4615536d4cbaf00efcc HTTP/1.0" 202 Accepted 2026-02-27 16:40:11,406 - lava_callback - INFO - Starting async job submit for node 69a1c4615536d4cbaf00efcc 2026-02-27 16:40:11,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:11,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c4615536d4cbaf00efcc HTTP/1.1" 200 None 2026-02-27 16:40:11,481 - lava_callback - DEBUG - Processing node 69a1c4615536d4cbaf00efcc: baseline-arm-clabbe 2026-02-27 16:40:11,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:40:11,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:40:11,802 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69a1c4615536d4cbaf00efcc/log.txt.gz 2026-02-27 16:40:11,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:40:12,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:40:12,122 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69a1c4615536d4cbaf00efcc/lava-logs.yaml 2026-02-27 16:40:12,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:40:12,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:40:12,271 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69a1c4615536d4cbaf00efcc/lava_callback.json.gz DEBUG: Setup failed for 69a1c4615536d4cbaf00efcc. Transitting job node result: incomplete -> fail 2026-02-27 16:40:12,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:12,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c4615536d4cbaf00efcc HTTP/1.1" 200 None 2026-02-27 16:40:12,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:12,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1b211d15a97544f0cb9d7 HTTP/1.1" 200 None 2026-02-27 16:40:12,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:12,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69a1c4615536d4cbaf00efcc HTTP/1.1" 200 None 2026-02-27 16:40:12,464 - lava_callback - INFO - Completed processing callback for node 69a1c4615536d4cbaf00efcc INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44156 - "GET /api/metrics HTTP/1.0" 200 OK 2026-02-27 16:40:16,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:16,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39536 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:43:03,441 - lava_callback - INFO - Received callback for node 699f2691cc090a0ddfeff67d INFO: 172.21.0.1:60164 - "POST /node/699f2691cc090a0ddfeff67d HTTP/1.0" 202 Accepted 2026-02-27 16:43:03,449 - lava_callback - INFO - Starting async job submit for node 699f2691cc090a0ddfeff67d 2026-02-27 16:43:03,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/699f2691cc090a0ddfeff67d HTTP/1.1" 200 None 2026-02-27 16:43:03,472 - lava_callback - DEBUG - Processing node 699f2691cc090a0ddfeff67d: baseline-arm64-broonie 2026-02-27 16:43:03,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 699f2691cc090a0ddfeff67d 2026-02-27 16:43:03,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/699f2691cc090a0ddfeff67d HTTP/1.1" 200 None 2026-02-27 16:43:03,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/699f2088cc090a0ddfeff153 HTTP/1.1" 200 None 2026-02-27 16:43:03,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/699f2691cc090a0ddfeff67d HTTP/1.1" 200 None 2026-02-27 16:43:03,547 - lava_callback - ERROR - Error processing callback for node 699f2691cc090a0ddfeff67d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1] INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:28:45,505 - lava_callback - INFO - Received callback for node 69a1be5669d5b6151c03ae23 2026-02-27 16:28:45,554 - lava_callback - INFO - Starting async job submit for node 69a1be5669d5b6151c03ae23 INFO: 172.21.0.1:47822 - "POST /node/69a1be5669d5b6151c03ae23 HTTP/1.0" 202 Accepted 2026-02-27 16:28:45,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:45,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1be5669d5b6151c03ae23 HTTP/1.1" 200 None 2026-02-27 16:28:45,600 - lava_callback - DEBUG - Processing node 69a1be5669d5b6151c03ae23: baseline-arm-pengutronix 2026-02-27 16:28:45,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:28:45,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:28:45,860 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69a1be5669d5b6151c03ae23/log.txt.gz 2026-02-27 16:28:45,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:28:46,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:28:46,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69a1be5669d5b6151c03ae23/lava-logs.yaml 2026-02-27 16:28:46,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:28:46,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:28:46,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69a1be5669d5b6151c03ae23/lava_callback.json.gz DEBUG: Setup failed for 69a1be5669d5b6151c03ae23. Transitting job node result: incomplete -> fail 2026-02-27 16:28:46,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:46,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1be5669d5b6151c03ae23 HTTP/1.1" 200 None 2026-02-27 16:28:46,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:46,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1b211d15a97544f0cb9d7 HTTP/1.1" 200 None 2026-02-27 16:28:46,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:28:46,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69a1be5669d5b6151c03ae23 HTTP/1.1" 200 None 2026-02-27 16:28:46,410 - lava_callback - INFO - Completed processing callback for node 69a1be5669d5b6151c03ae23 INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34076 - "GET /api/metrics HTTP/1.0" 200 OK 2026-02-27 16:29:16,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:16,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33932 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:29:37,141 - lava_callback - INFO - Received callback for node 69a1c27c5536d4cbaf00ef64 INFO: 172.21.0.1:33934 - "POST /node/69a1c27c5536d4cbaf00ef64 HTTP/1.0" 202 Accepted 2026-02-27 16:29:37,148 - lava_callback - INFO - Starting async job submit for node 69a1c27c5536d4cbaf00ef64 2026-02-27 16:29:37,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:37,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c27c5536d4cbaf00ef64 HTTP/1.1" 200 None 2026-02-27 16:29:37,220 - lava_callback - DEBUG - Processing node 69a1c27c5536d4cbaf00ef64: baseline-arm 2026-02-27 16:29:37,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:29:37,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:29:37,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69a1c27c5536d4cbaf00ef64/log.txt.gz 2026-02-27 16:29:38,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:29:38,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:29:38,584 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69a1c27c5536d4cbaf00ef64/lava-logs.yaml 2026-02-27 16:29:38,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:29:38,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:29:38,792 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69a1c27c5536d4cbaf00ef64/lava_callback.json.gz DEBUG: Setup failed for 69a1c27c5536d4cbaf00ef64. Transitting job node result: incomplete -> fail 2026-02-27 16:29:38,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:38,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c27c5536d4cbaf00ef64 HTTP/1.1" 200 None 2026-02-27 16:29:38,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:38,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1b211d15a97544f0cb9d7 HTTP/1.1" 200 None 2026-02-27 16:29:38,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:38,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69a1c27c5536d4cbaf00ef64 HTTP/1.1" 200 None 2026-02-27 16:29:39,069 - lava_callback - INFO - Completed processing callback for node 69a1c27c5536d4cbaf00ef64 2026-02-27 16:29:46,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:29:46,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39008 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:40:11,400 - lava_callback - INFO - Received callback for node 69a1c4615536d4cbaf00efcc INFO: 172.21.0.1:44146 - "POST /node/69a1c4615536d4cbaf00efcc HTTP/1.0" 202 Accepted 2026-02-27 16:40:11,406 - lava_callback - INFO - Starting async job submit for node 69a1c4615536d4cbaf00efcc 2026-02-27 16:40:11,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:11,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c4615536d4cbaf00efcc HTTP/1.1" 200 None 2026-02-27 16:40:11,481 - lava_callback - DEBUG - Processing node 69a1c4615536d4cbaf00efcc: baseline-arm-clabbe 2026-02-27 16:40:11,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:40:11,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:40:11,802 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69a1c4615536d4cbaf00efcc/log.txt.gz 2026-02-27 16:40:11,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:40:12,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:40:12,122 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69a1c4615536d4cbaf00efcc/lava-logs.yaml 2026-02-27 16:40:12,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-02-27 16:40:12,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-02-27 16:40:12,271 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69a1c4615536d4cbaf00efcc/lava_callback.json.gz DEBUG: Setup failed for 69a1c4615536d4cbaf00efcc. Transitting job node result: incomplete -> fail 2026-02-27 16:40:12,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:12,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1c4615536d4cbaf00efcc HTTP/1.1" 200 None 2026-02-27 16:40:12,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:12,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a1b211d15a97544f0cb9d7 HTTP/1.1" 200 None 2026-02-27 16:40:12,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:12,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69a1c4615536d4cbaf00efcc HTTP/1.1" 200 None 2026-02-27 16:40:12,464 - lava_callback - INFO - Completed processing callback for node 69a1c4615536d4cbaf00efcc INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44156 - "GET /api/metrics HTTP/1.0" 200 OK 2026-02-27 16:40:16,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:40:16,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39536 - "GET / HTTP/1.0" 200 OK 2026-02-27 16:43:03,441 - lava_callback - INFO - Received callback for node 699f2691cc090a0ddfeff67d INFO: 172.21.0.1:60164 - "POST /node/699f2691cc090a0ddfeff67d HTTP/1.0" 202 Accepted 2026-02-27 16:43:03,449 - lava_callback - INFO - Starting async job submit for node 699f2691cc090a0ddfeff67d 2026-02-27 16:43:03,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/699f2691cc090a0ddfeff67d HTTP/1.1" 200 None 2026-02-27 16:43:03,472 - lava_callback - DEBUG - Processing node 699f2691cc090a0ddfeff67d: baseline-arm64-broonie 2026-02-27 16:43:03,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 699f2691cc090a0ddfeff67d 2026-02-27 16:43:03,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/699f2691cc090a0ddfeff67d HTTP/1.1" 200 None 2026-02-27 16:43:03,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/699f2088cc090a0ddfeff153 HTTP/1.1" 200 None 2026-02-27 16:43:03,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-02-27 16:43:03,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/699f2691cc090a0ddfeff67d HTTP/1.1" 200 None 2026-02-27 16:43:03,547 - lava_callback - ERROR - Error processing callback for node 699f2691cc090a0ddfeff67d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]