INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57602 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:30:06,331 - lava_callback - INFO - Received callback for node 69c7911cbe430e18c5d15fde 2026-03-28 08:30:06,345 - lava_callback - INFO - Starting async job submit for node 69c7911cbe430e18c5d15fde INFO: 172.20.0.1:48574 - "POST /node/69c7911cbe430e18c5d15fde HTTP/1.0" 202 Accepted 2026-03-28 08:30:06,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:30:06,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7911cbe430e18c5d15fde HTTP/1.1" 200 None 2026-03-28 08:30:06,399 - lava_callback - DEBUG - Processing node 69c7911cbe430e18c5d15fde: baseline-arm INFO: 172.20.0.1:48582 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:30:06,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:30:06,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:30:06,845 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7911cbe430e18c5d15fde/log.txt.gz 2026-03-28 08:30:06,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:30:07,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:30:07,164 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7911cbe430e18c5d15fde/lava-logs.yaml 2026-03-28 08:30:07,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:30:07,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:30:07,332 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7911cbe430e18c5d15fde/lava_callback.json.gz 2026-03-28 08:30:07,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:30:07,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7911cbe430e18c5d15fde HTTP/1.1" 200 None 2026-03-28 08:30:07,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:30:07,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb1be430e18c5d159e5 HTTP/1.1" 200 None 2026-03-28 08:30:07,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:30:07,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7911cbe430e18c5d15fde HTTP/1.1" 200 None 2026-03-28 08:30:07,426 - lava_callback - INFO - Completed processing callback for node 69c7911cbe430e18c5d15fde INFO: 172.20.0.1:53888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51906 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:30:37,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:30:37,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35878 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:39:21,237 - lava_callback - INFO - Received callback for node 69c793a2be430e18c5d16004 INFO: 172.20.0.1:38450 - "POST /node/69c793a2be430e18c5d16004 HTTP/1.0" 202 Accepted 2026-03-28 08:39:21,248 - lava_callback - INFO - Starting async job submit for node 69c793a2be430e18c5d16004 2026-03-28 08:39:21,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:21,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a2be430e18c5d16004 HTTP/1.1" 200 None 2026-03-28 08:39:21,287 - lava_callback - DEBUG - Processing node 69c793a2be430e18c5d16004: baseline-arm 2026-03-28 08:39:21,295 - lava_callback - WARNING - Failed to upload log 2026-03-28 08:39:21,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:39:21,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:21,497 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a2be430e18c5d16004/lava-logs.yaml 2026-03-28 08:39:21,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:39:21,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:21,623 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a2be430e18c5d16004/lava_callback.json.gz 2026-03-28 08:39:21,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:21,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a2be430e18c5d16004 HTTP/1.1" 200 None 2026-03-28 08:39:21,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:21,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:39:21,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:21,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793a2be430e18c5d16004 HTTP/1.1" 200 None 2026-03-28 08:39:21,687 - lava_callback - INFO - Completed processing callback for node 69c793a2be430e18c5d16004 INFO: 172.20.0.1:38456 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:39:36,259 - lava_callback - INFO - Received callback for node 69c793a6be430e18c5d16010 INFO: 172.20.0.1:39624 - "POST /node/69c793a6be430e18c5d16010 HTTP/1.0" 202 Accepted 2026-03-28 08:39:36,306 - lava_callback - INFO - Starting async job submit for node 69c793a6be430e18c5d16010 2026-03-28 08:39:36,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:36,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a6be430e18c5d16010 HTTP/1.1" 200 None 2026-03-28 08:39:36,342 - lava_callback - DEBUG - Processing node 69c793a6be430e18c5d16010: baseline-arm 2026-03-28 08:39:36,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39628 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:39:36,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:36,596 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a6be430e18c5d16010/log.txt.gz 2026-03-28 08:39:36,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:39:36,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:36,870 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a6be430e18c5d16010/lava-logs.yaml 2026-03-28 08:39:36,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:39:36,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:36,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a6be430e18c5d16010/lava_callback.json.gz 2026-03-28 08:39:37,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:37,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a6be430e18c5d16010 HTTP/1.1" 200 None 2026-03-28 08:39:37,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:37,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:39:37,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:37,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793a6be430e18c5d16010 HTTP/1.1" 200 None 2026-03-28 08:39:37,068 - lava_callback - INFO - Completed processing callback for node 69c793a6be430e18c5d16010 2026-03-28 08:39:37,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:37,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 08:39:38,645 - lava_callback - INFO - Received callback for node 69c793a3be430e18c5d16007 INFO: 172.20.0.1:39638 - "POST /node/69c793a3be430e18c5d16007 HTTP/1.0" 202 Accepted 2026-03-28 08:39:38,698 - lava_callback - INFO - Starting async job submit for node 69c793a3be430e18c5d16007 2026-03-28 08:39:38,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:38,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a3be430e18c5d16007 HTTP/1.1" 200 None 2026-03-28 08:39:38,745 - lava_callback - DEBUG - Processing node 69c793a3be430e18c5d16007: baseline-arm 2026-03-28 08:39:38,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:39:38,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:39,001 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a3be430e18c5d16007/log.txt.gz 2026-03-28 08:39:39,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:39:39,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:39,248 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a3be430e18c5d16007/lava-logs.yaml 2026-03-28 08:39:39,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:39:39,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:39:39,369 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a3be430e18c5d16007/lava_callback.json.gz 2026-03-28 08:39:39,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:39,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a3be430e18c5d16007 HTTP/1.1" 200 None 2026-03-28 08:39:39,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:39,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:39:39,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:39:39,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793a3be430e18c5d16007 HTTP/1.1" 200 None 2026-03-28 08:39:39,443 - lava_callback - INFO - Completed processing callback for node 69c793a3be430e18c5d16007 INFO: 172.20.0.1:56080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40924 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:40:07,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:07,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 08:40:11,899 - lava_callback - INFO - Received callback for node 69c4f0e1f63ce05331e2aae1 INFO: 172.20.0.1:36928 - "POST /node/69c4f0e1f63ce05331e2aae1 HTTP/1.0" 202 Accepted 2026-03-28 08:40:11,908 - lava_callback - INFO - Starting async job submit for node 69c4f0e1f63ce05331e2aae1 2026-03-28 08:40:11,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:11,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e1f63ce05331e2aae1 HTTP/1.1" 200 None 2026-03-28 08:40:11,933 - lava_callback - DEBUG - Processing node 69c4f0e1f63ce05331e2aae1: baseline-arm-baylibre 2026-03-28 08:40:11,934 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f0e1f63ce05331e2aae1 2026-03-28 08:40:11,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:11,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0e1f63ce05331e2aae1 HTTP/1.1" 200 None 2026-03-28 08:40:11,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:11,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 08:40:11,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:12,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0e1f63ce05331e2aae1 HTTP/1.1" 200 None 2026-03-28 08:40:12,030 - lava_callback - ERROR - Error processing callback for node 69c4f0e1f63ce05331e2aae1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36946 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:40:22,874 - lava_callback - INFO - Received callback for node 69c4f0edf63ce05331e2ab02 INFO: 172.20.0.1:38670 - "POST /node/69c4f0edf63ce05331e2ab02 HTTP/1.0" 202 Accepted 2026-03-28 08:40:22,887 - lava_callback - INFO - Starting async job submit for node 69c4f0edf63ce05331e2ab02 2026-03-28 08:40:22,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:22,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0edf63ce05331e2ab02 HTTP/1.1" 200 None 2026-03-28 08:40:22,914 - lava_callback - DEBUG - Processing node 69c4f0edf63ce05331e2ab02: baseline-arm-clabbe 2026-03-28 08:40:22,914 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f0edf63ce05331e2ab02 2026-03-28 08:40:22,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:22,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0edf63ce05331e2ab02 HTTP/1.1" 200 None 2026-03-28 08:40:22,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:22,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 08:40:22,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:23,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0edf63ce05331e2ab02 HTTP/1.1" 200 None 2026-03-28 08:40:23,016 - lava_callback - ERROR - Error processing callback for node 69c4f0edf63ce05331e2ab02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:40:23,514 - lava_callback - INFO - Received callback for node 69c4f0f1f63ce05331e2ab15 INFO: 172.20.0.1:38672 - "POST /node/69c4f0f1f63ce05331e2ab15 HTTP/1.0" 202 Accepted 2026-03-28 08:40:23,522 - lava_callback - INFO - Starting async job submit for node 69c4f0f1f63ce05331e2ab15 2026-03-28 08:40:23,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:23,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f1f63ce05331e2ab15 HTTP/1.1" 200 None 2026-03-28 08:40:23,554 - lava_callback - DEBUG - Processing node 69c4f0f1f63ce05331e2ab15: baseline-arm-clabbe 2026-03-28 08:40:23,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f0f1f63ce05331e2ab15 2026-03-28 08:40:23,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:23,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f1f63ce05331e2ab15 HTTP/1.1" 200 None 2026-03-28 08:40:23,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:23,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 08:40:23,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:23,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f1f63ce05331e2ab15 HTTP/1.1" 200 None 2026-03-28 08:40:23,643 - lava_callback - ERROR - Error processing callback for node 69c4f0f1f63ce05331e2ab15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:40:24,178 - lava_callback - INFO - Received callback for node 69c4f0f2f63ce05331e2ab18 INFO: 172.20.0.1:38686 - "POST /node/69c4f0f2f63ce05331e2ab18 HTTP/1.0" 202 Accepted 2026-03-28 08:40:24,187 - lava_callback - INFO - Starting async job submit for node 69c4f0f2f63ce05331e2ab18 2026-03-28 08:40:24,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:24,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f2f63ce05331e2ab18 HTTP/1.1" 200 None 2026-03-28 08:40:24,219 - lava_callback - DEBUG - Processing node 69c4f0f2f63ce05331e2ab18: baseline-arm-clabbe 2026-03-28 08:40:24,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f0f2f63ce05331e2ab18 2026-03-28 08:40:24,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:24,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f2f63ce05331e2ab18 HTTP/1.1" 200 None 2026-03-28 08:40:24,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:24,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 08:40:24,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:24,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f2f63ce05331e2ab18 HTTP/1.1" 200 None 2026-03-28 08:40:24,320 - lava_callback - ERROR - Error processing callback for node 69c4f0f2f63ce05331e2ab18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:40:24,700 - lava_callback - INFO - Received callback for node 69c793a5be430e18c5d1600d INFO: 172.20.0.1:38694 - "POST /node/69c793a5be430e18c5d1600d HTTP/1.0" 202 Accepted 2026-03-28 08:40:24,709 - lava_callback - INFO - Starting async job submit for node 69c793a5be430e18c5d1600d 2026-03-28 08:40:24,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:24,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a5be430e18c5d1600d HTTP/1.1" 200 None 2026-03-28 08:40:24,749 - lava_callback - DEBUG - Processing node 69c793a5be430e18c5d1600d: baseline-arm 2026-03-28 08:40:24,859 - lava_callback - INFO - Received callback for node 69c4f0f3f63ce05331e2ab1b 2026-03-28 08:40:24,902 - lava_callback - INFO - Starting async job submit for node 69c4f0f3f63ce05331e2ab1b 2026-03-28 08:40:24,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:38696 - "POST /node/69c4f0f3f63ce05331e2ab1b HTTP/1.0" 202 Accepted 2026-03-28 08:40:24,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f3f63ce05331e2ab1b HTTP/1.1" 200 None 2026-03-28 08:40:24,991 - lava_callback - DEBUG - Processing node 69c4f0f3f63ce05331e2ab1b: baseline-arm-clabbe 2026-03-28 08:40:24,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f0f3f63ce05331e2ab1b 2026-03-28 08:40:24,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:25,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f3f63ce05331e2ab1b HTTP/1.1" 200 None 2026-03-28 08:40:25,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:25,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 08:40:25,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:25,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f3f63ce05331e2ab1b HTTP/1.1" 200 None 2026-03-28 08:40:25,224 - lava_callback - ERROR - Error processing callback for node 69c4f0f3f63ce05331e2ab1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:40:25,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:25,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:25,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a5be430e18c5d1600d/log.txt.gz 2026-03-28 08:40:25,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:26,036 - lava_callback - INFO - Received callback for node 69c793a1be430e18c5d16001 INFO: 172.20.0.1:38700 - "POST /node/69c793a1be430e18c5d16001 HTTP/1.0" 202 Accepted 2026-03-28 08:40:26,048 - lava_callback - INFO - Starting async job submit for node 69c793a1be430e18c5d16001 2026-03-28 08:40:26,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:26,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a1be430e18c5d16001 HTTP/1.1" 200 None 2026-03-28 08:40:26,109 - lava_callback - DEBUG - Processing node 69c793a1be430e18c5d16001: baseline-arm 2026-03-28 08:40:26,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:26,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:26,449 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a5be430e18c5d1600d/lava-logs.yaml 2026-03-28 08:40:26,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:38702 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:40:26,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:26,592 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a1be430e18c5d16001/log.txt.gz 2026-03-28 08:40:26,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:26,644 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a5be430e18c5d1600d/lava_callback.json.gz 2026-03-28 08:40:26,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:26,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:26,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a5be430e18c5d1600d HTTP/1.1" 200 None 2026-03-28 08:40:26,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:26,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:26,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:26,829 - lava_callback - INFO - Received callback for node 69c793aebe430e18c5d1602b 2026-03-28 08:40:26,838 - lava_callback - INFO - Starting async job submit for node 69c793aebe430e18c5d1602b INFO: 172.20.0.1:38718 - "POST /node/69c793aebe430e18c5d1602b HTTP/1.0" 202 Accepted 2026-03-28 08:40:26,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793a5be430e18c5d1600d HTTP/1.1" 200 None 2026-03-28 08:40:26,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:26,914 - lava_callback - INFO - Completed processing callback for node 69c793a5be430e18c5d1600d 2026-03-28 08:40:26,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793aebe430e18c5d1602b HTTP/1.1" 200 None 2026-03-28 08:40:26,953 - lava_callback - DEBUG - Processing node 69c793aebe430e18c5d1602b: baseline-arm-broonie 2026-03-28 08:40:27,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:27,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:27,447 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a1be430e18c5d16001/lava-logs.yaml 2026-03-28 08:40:27,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:27,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:27,551 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793aebe430e18c5d1602b/log.txt.gz 2026-03-28 08:40:27,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:27,655 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a1be430e18c5d16001/lava_callback.json.gz 2026-03-28 08:40:27,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:27,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:27,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a1be430e18c5d16001 HTTP/1.1" 200 None 2026-03-28 08:40:27,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:27,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:27,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:27,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793a1be430e18c5d16001 HTTP/1.1" 200 None 2026-03-28 08:40:27,881 - lava_callback - INFO - Completed processing callback for node 69c793a1be430e18c5d16001 2026-03-28 08:40:28,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:28,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793aebe430e18c5d1602b/lava-logs.yaml 2026-03-28 08:40:28,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:28,298 - lava_callback - INFO - Received callback for node 69c793b0be430e18c5d16031 INFO: 172.20.0.1:38724 - "POST /node/69c793b0be430e18c5d16031 HTTP/1.0" 202 Accepted 2026-03-28 08:40:28,305 - lava_callback - INFO - Starting async job submit for node 69c793b0be430e18c5d16031 2026-03-28 08:40:28,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:28,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b0be430e18c5d16031 HTTP/1.1" 200 None 2026-03-28 08:40:28,354 - lava_callback - DEBUG - Processing node 69c793b0be430e18c5d16031: baseline-arm-broonie 2026-03-28 08:40:28,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:28,435 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793aebe430e18c5d1602b/lava_callback.json.gz 2026-03-28 08:40:28,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:28,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793aebe430e18c5d1602b HTTP/1.1" 200 None 2026-03-28 08:40:28,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:28,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:28,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:28,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793aebe430e18c5d1602b HTTP/1.1" 200 None 2026-03-28 08:40:28,734 - lava_callback - INFO - Completed processing callback for node 69c793aebe430e18c5d1602b 2026-03-28 08:40:28,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:29,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:29,111 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793b0be430e18c5d16031/log.txt.gz 2026-03-28 08:40:29,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:29,457 - lava_callback - INFO - Received callback for node 69c793bfbe430e18c5d16067 INFO: 172.20.0.1:38732 - "POST /node/69c793bfbe430e18c5d16067 HTTP/1.0" 202 Accepted 2026-03-28 08:40:29,477 - lava_callback - INFO - Starting async job submit for node 69c793bfbe430e18c5d16067 2026-03-28 08:40:29,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:29,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793bfbe430e18c5d16067 HTTP/1.1" 200 None 2026-03-28 08:40:29,555 - lava_callback - DEBUG - Processing node 69c793bfbe430e18c5d16067: baseline-arm-pengutronix 2026-03-28 08:40:29,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:29,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:29,870 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793b0be430e18c5d16031/lava-logs.yaml 2026-03-28 08:40:29,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:30,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:30,007 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793bfbe430e18c5d16067/log.txt.gz 2026-03-28 08:40:30,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:30,097 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793b0be430e18c5d16031/lava_callback.json.gz 2026-03-28 08:40:30,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:30,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:30,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b0be430e18c5d16031 HTTP/1.1" 200 None 2026-03-28 08:40:30,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:30,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:30,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:30,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b0be430e18c5d16031 HTTP/1.1" 200 None 2026-03-28 08:40:30,209 - lava_callback - INFO - Completed processing callback for node 69c793b0be430e18c5d16031 2026-03-28 08:40:30,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:30,695 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793bfbe430e18c5d16067/lava-logs.yaml 2026-03-28 08:40:30,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:30,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:30,978 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793bfbe430e18c5d16067/lava_callback.json.gz 2026-03-28 08:40:30,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:31,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793bfbe430e18c5d16067 HTTP/1.1" 200 None 2026-03-28 08:40:31,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:31,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:31,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:31,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793bfbe430e18c5d16067 HTTP/1.1" 200 None 2026-03-28 08:40:31,064 - lava_callback - INFO - Completed processing callback for node 69c793bfbe430e18c5d16067 2026-03-28 08:40:34,157 - lava_callback - INFO - Received callback for node 69c793c3be430e18c5d16070 INFO: 172.20.0.1:55894 - "POST /node/69c793c3be430e18c5d16070 HTTP/1.0" 202 Accepted 2026-03-28 08:40:34,211 - lava_callback - INFO - Starting async job submit for node 69c793c3be430e18c5d16070 2026-03-28 08:40:34,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:34,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793c3be430e18c5d16070 HTTP/1.1" 200 None 2026-03-28 08:40:34,257 - lava_callback - DEBUG - Processing node 69c793c3be430e18c5d16070: baseline-arm-pengutronix 2026-03-28 08:40:34,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:34,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:34,845 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c3be430e18c5d16070/log.txt.gz 2026-03-28 08:40:34,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:35,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:35,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c3be430e18c5d16070/lava-logs.yaml 2026-03-28 08:40:35,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:35,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:35,519 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c3be430e18c5d16070/lava_callback.json.gz 2026-03-28 08:40:35,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:35,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793c3be430e18c5d16070 HTTP/1.1" 200 None 2026-03-28 08:40:35,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:35,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:35,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:35,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793c3be430e18c5d16070 HTTP/1.1" 200 None 2026-03-28 08:40:35,639 - lava_callback - INFO - Completed processing callback for node 69c793c3be430e18c5d16070 INFO: 172.20.0.1:55902 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:40:37,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:37,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:57320 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:40:43,004 - lava_callback - INFO - Received callback for node 69c4f0f5f63ce05331e2ab53 INFO: 172.20.0.1:57330 - "POST /node/69c4f0f5f63ce05331e2ab53 HTTP/1.0" 202 Accepted 2026-03-28 08:40:43,010 - lava_callback - INFO - Starting async job submit for node 69c4f0f5f63ce05331e2ab53 2026-03-28 08:40:43,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:43,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f5f63ce05331e2ab53 HTTP/1.1" 200 None 2026-03-28 08:40:43,026 - lava_callback - DEBUG - Processing node 69c4f0f5f63ce05331e2ab53: baseline-arm-clabbe 2026-03-28 08:40:43,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f0f5f63ce05331e2ab53 2026-03-28 08:40:43,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:43,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f0f5f63ce05331e2ab53 HTTP/1.1" 200 None 2026-03-28 08:40:43,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:43,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 08:40:43,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:43,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f0f5f63ce05331e2ab53 HTTP/1.1" 200 None 2026-03-28 08:40:43,084 - lava_callback - ERROR - Error processing callback for node 69c4f0f5f63ce05331e2ab53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57344 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:40:49,864 - lava_callback - INFO - Received callback for node 69c793b5be430e18c5d16045 INFO: 172.20.0.1:40092 - "POST /node/69c793b5be430e18c5d16045 HTTP/1.0" 202 Accepted 2026-03-28 08:40:49,874 - lava_callback - INFO - Starting async job submit for node 69c793b5be430e18c5d16045 2026-03-28 08:40:49,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:49,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b5be430e18c5d16045 HTTP/1.1" 200 None 2026-03-28 08:40:49,943 - lava_callback - DEBUG - Processing node 69c793b5be430e18c5d16045: baseline-arm-clabbe 2026-03-28 08:40:50,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:50,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:50,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793b5be430e18c5d16045/log.txt.gz 2026-03-28 08:40:50,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:51,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:51,050 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793b5be430e18c5d16045/lava-logs.yaml 2026-03-28 08:40:51,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:51,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:51,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793b5be430e18c5d16045/lava_callback.json.gz 2026-03-28 08:40:51,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:51,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b5be430e18c5d16045 HTTP/1.1" 200 None 2026-03-28 08:40:51,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:51,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:51,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:51,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b5be430e18c5d16045 HTTP/1.1" 200 None 2026-03-28 08:40:51,274 - lava_callback - INFO - Completed processing callback for node 69c793b5be430e18c5d16045 2026-03-28 08:40:55,454 - lava_callback - INFO - Received callback for node 69c793b1be430e18c5d16035 INFO: 172.20.0.1:40102 - "POST /node/69c793b1be430e18c5d16035 HTTP/1.0" 202 Accepted 2026-03-28 08:40:55,462 - lava_callback - INFO - Starting async job submit for node 69c793b1be430e18c5d16035 2026-03-28 08:40:55,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:55,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b1be430e18c5d16035 HTTP/1.1" 200 None 2026-03-28 08:40:55,501 - lava_callback - DEBUG - Processing node 69c793b1be430e18c5d16035: baseline-arm-broonie 2026-03-28 08:40:55,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:56,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:56,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793b1be430e18c5d16035/log.txt.gz 2026-03-28 08:40:56,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:40:56,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:56,467 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793b1be430e18c5d16035/lava-logs.yaml 2026-03-28 08:40:56,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40112 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:40:56,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:40:56,823 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793b1be430e18c5d16035/lava_callback.json.gz 2026-03-28 08:40:56,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:56,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b1be430e18c5d16035 HTTP/1.1" 200 None 2026-03-28 08:40:56,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:56,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:40:56,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:40:56,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b1be430e18c5d16035 HTTP/1.1" 200 None 2026-03-28 08:40:56,953 - lava_callback - INFO - Completed processing callback for node 69c793b1be430e18c5d16035 INFO: 172.20.0.1:34122 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:41:07,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:07,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:41:38,842 - lava_callback - INFO - Received callback for node 69c793c0be430e18c5d1606a INFO: 172.20.0.1:37608 - "POST /node/69c793c0be430e18c5d1606a HTTP/1.0" 202 Accepted 2026-03-28 08:41:38,848 - lava_callback - INFO - Starting async job submit for node 69c793c0be430e18c5d1606a 2026-03-28 08:41:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:38,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793c0be430e18c5d1606a HTTP/1.1" 200 None 2026-03-28 08:41:38,905 - lava_callback - DEBUG - Processing node 69c793c0be430e18c5d1606a: baseline-arm-pengutronix 2026-03-28 08:41:39,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:39,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:39,199 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c0be430e18c5d1606a/log.txt.gz 2026-03-28 08:41:39,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:39,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:39,647 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c0be430e18c5d1606a/lava-logs.yaml 2026-03-28 08:41:39,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:39,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:39,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c0be430e18c5d1606a/lava_callback.json.gz 2026-03-28 08:41:39,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:39,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793c0be430e18c5d1606a HTTP/1.1" 200 None 2026-03-28 08:41:39,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:39,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:41:39,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:39,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793c0be430e18c5d1606a HTTP/1.1" 200 None 2026-03-28 08:41:39,960 - lava_callback - INFO - Completed processing callback for node 69c793c0be430e18c5d1606a INFO: 172.20.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58100 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:41:46,789 - lava_callback - INFO - Received callback for node 69c793dabe430e18c5d160c1 INFO: 172.20.0.1:58104 - "POST /node/69c793dabe430e18c5d160c1 HTTP/1.0" 202 Accepted 2026-03-28 08:41:46,796 - lava_callback - INFO - Starting async job submit for node 69c793dabe430e18c5d160c1 2026-03-28 08:41:46,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:46,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793dabe430e18c5d160c1 HTTP/1.1" 200 None 2026-03-28 08:41:46,857 - lava_callback - DEBUG - Processing node 69c793dabe430e18c5d160c1: ltp-dio 2026-03-28 08:41:47,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:47,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:47,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c793dabe430e18c5d160c1/log.txt.gz 2026-03-28 08:41:47,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:47,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:47,813 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c793dabe430e18c5d160c1/lava-logs.yaml 2026-03-28 08:41:47,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:47,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:47,976 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c793dabe430e18c5d160c1/lava_callback.json.gz 2026-03-28 08:41:48,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:48,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793dabe430e18c5d160c1 HTTP/1.1" 200 None 2026-03-28 08:41:48,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:48,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:41:48,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:48,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793dabe430e18c5d160c1 HTTP/1.1" 200 None 2026-03-28 08:41:48,058 - lava_callback - INFO - Completed processing callback for node 69c793dabe430e18c5d160c1 2026-03-28 08:41:55,392 - lava_callback - INFO - Received callback for node 69c793adbe430e18c5d16028 INFO: 172.20.0.1:58842 - "POST /node/69c793adbe430e18c5d16028 HTTP/1.0" 202 Accepted 2026-03-28 08:41:55,398 - lava_callback - INFO - Starting async job submit for node 69c793adbe430e18c5d16028 2026-03-28 08:41:55,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:55,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793adbe430e18c5d16028 HTTP/1.1" 200 None 2026-03-28 08:41:55,434 - lava_callback - DEBUG - Processing node 69c793adbe430e18c5d16028: baseline-arm-broonie 2026-03-28 08:41:55,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:55,780 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:55,782 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793adbe430e18c5d16028/log.txt.gz 2026-03-28 08:41:55,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:56,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:56,097 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793adbe430e18c5d16028/lava-logs.yaml 2026-03-28 08:41:56,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:56,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:56,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c793adbe430e18c5d16028/lava_callback.json.gz 2026-03-28 08:41:56,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:56,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793adbe430e18c5d16028 HTTP/1.1" 200 None 2026-03-28 08:41:56,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:56,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:41:56,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:56,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793adbe430e18c5d16028 HTTP/1.1" 200 None 2026-03-28 08:41:56,326 - lava_callback - INFO - Completed processing callback for node 69c793adbe430e18c5d16028 INFO: 172.20.0.1:58856 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:41:57,021 - lava_callback - INFO - Received callback for node 69c79441be430e18c5d16168 INFO: 172.20.0.1:58860 - "POST /node/69c79441be430e18c5d16168 HTTP/1.0" 202 Accepted 2026-03-28 08:41:57,027 - lava_callback - INFO - Starting async job submit for node 69c79441be430e18c5d16168 2026-03-28 08:41:57,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:57,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79441be430e18c5d16168 HTTP/1.1" 200 None 2026-03-28 08:41:57,045 - lava_callback - DEBUG - Processing node 69c79441be430e18c5d16168: baseline-arm 2026-03-28 08:41:57,050 - lava_callback - WARNING - Failed to upload log 2026-03-28 08:41:57,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:57,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:57,189 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c79441be430e18c5d16168/lava-logs.yaml 2026-03-28 08:41:57,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:41:57,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:41:57,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c79441be430e18c5d16168/lava_callback.json.gz 2026-03-28 08:41:57,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:57,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79441be430e18c5d16168 HTTP/1.1" 200 None 2026-03-28 08:41:57,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:57,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:41:57,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:41:57,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79441be430e18c5d16168 HTTP/1.1" 200 None 2026-03-28 08:41:57,369 - lava_callback - INFO - Completed processing callback for node 69c79441be430e18c5d16168 INFO: 172.20.0.1:41824 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:42:07,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:42:07,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:46112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38296 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:42:41,028 - lava_callback - INFO - Received callback for node 69c793b6be430e18c5d16048 INFO: 172.20.0.1:44232 - "POST /node/69c793b6be430e18c5d16048 HTTP/1.0" 202 Accepted 2026-03-28 08:42:41,034 - lava_callback - INFO - Starting async job submit for node 69c793b6be430e18c5d16048 2026-03-28 08:42:41,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:42:41,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b6be430e18c5d16048 HTTP/1.1" 200 None 2026-03-28 08:42:41,071 - lava_callback - DEBUG - Processing node 69c793b6be430e18c5d16048: baseline-arm-clabbe 2026-03-28 08:42:41,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:42:41,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:42:41,439 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793b6be430e18c5d16048/log.txt.gz 2026-03-28 08:42:41,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:42:41,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:42:41,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793b6be430e18c5d16048/lava-logs.yaml 2026-03-28 08:42:41,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:42:42,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:42:42,102 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793b6be430e18c5d16048/lava_callback.json.gz 2026-03-28 08:42:42,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:42:42,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793b6be430e18c5d16048 HTTP/1.1" 200 None 2026-03-28 08:42:42,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:42:42,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:42:42,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:42:42,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793b6be430e18c5d16048 HTTP/1.1" 200 None 2026-03-28 08:42:42,182 - lava_callback - INFO - Completed processing callback for node 69c793b6be430e18c5d16048 INFO: 172.20.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33026 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:43:07,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:07,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52496 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:43:47,372 - lava_callback - INFO - Received callback for node 69c793d3be430e18c5d160b1 INFO: 172.20.0.1:52498 - "POST /node/69c793d3be430e18c5d160b1 HTTP/1.0" 202 Accepted 2026-03-28 08:43:47,382 - lava_callback - INFO - Starting async job submit for node 69c793d3be430e18c5d160b1 2026-03-28 08:43:47,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:47,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d3be430e18c5d160b1 HTTP/1.1" 200 None 2026-03-28 08:43:47,455 - lava_callback - DEBUG - Processing node 69c793d3be430e18c5d160b1: ltp-cap_bounds 2026-03-28 08:43:48,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:43:48,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:48,437 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c793d3be430e18c5d160b1/log.txt.gz 2026-03-28 08:43:48,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:43:49,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:49,377 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c793d3be430e18c5d160b1/lava-logs.yaml 2026-03-28 08:43:49,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:43:49,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:49,740 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c793d3be430e18c5d160b1/lava_callback.json.gz 2026-03-28 08:43:49,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:49,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d3be430e18c5d160b1 HTTP/1.1" 200 None 2026-03-28 08:43:49,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:49,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:43:49,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:49,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793d3be430e18c5d160b1 HTTP/1.1" 200 None 2026-03-28 08:43:49,874 - lava_callback - INFO - Completed processing callback for node 69c793d3be430e18c5d160b1 2026-03-28 08:43:55,709 - lava_callback - INFO - Received callback for node 69c793d5be430e18c5d160b8 INFO: 172.20.0.1:36142 - "POST /node/69c793d5be430e18c5d160b8 HTTP/1.0" 202 Accepted 2026-03-28 08:43:55,719 - lava_callback - INFO - Starting async job submit for node 69c793d5be430e18c5d160b8 2026-03-28 08:43:55,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:55,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d5be430e18c5d160b8 HTTP/1.1" 200 None 2026-03-28 08:43:55,800 - lava_callback - DEBUG - Processing node 69c793d5be430e18c5d160b8: ltp-containers 2026-03-28 08:43:56,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36152 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:43:56,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:56,962 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c793d5be430e18c5d160b8/log.txt.gz 2026-03-28 08:43:57,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:43:57,326 - lava_callback - INFO - Received callback for node 69c793d6be430e18c5d160bb INFO: 172.20.0.1:36156 - "POST /node/69c793d6be430e18c5d160bb HTTP/1.0" 202 Accepted 2026-03-28 08:43:57,337 - lava_callback - INFO - Starting async job submit for node 69c793d6be430e18c5d160bb 2026-03-28 08:43:57,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:57,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d6be430e18c5d160bb HTTP/1.1" 200 None 2026-03-28 08:43:57,401 - lava_callback - DEBUG - Processing node 69c793d6be430e18c5d160bb: ltp-controllers 2026-03-28 08:43:57,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:57,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c793d5be430e18c5d160b8/lava-logs.yaml 2026-03-28 08:43:58,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:43:58,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:43:58,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:58,512 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c793d5be430e18c5d160b8/lava_callback.json.gz 2026-03-28 08:43:58,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:58,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:58,556 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c793d6be430e18c5d160bb/log.txt.gz 2026-03-28 08:43:58,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d5be430e18c5d160b8 HTTP/1.1" 200 None 2026-03-28 08:43:58,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:58,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:43:58,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:43:58,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793d5be430e18c5d160b8 HTTP/1.1" 200 None 2026-03-28 08:43:58,872 - lava_callback - INFO - Completed processing callback for node 69c793d5be430e18c5d160b8 2026-03-28 08:43:59,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:43:59,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:43:59,724 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c793d6be430e18c5d160bb/lava-logs.yaml 2026-03-28 08:43:59,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:44:00,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:44:00,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c793d6be430e18c5d160bb/lava_callback.json.gz 2026-03-28 08:44:00,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:00,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d6be430e18c5d160bb HTTP/1.1" 200 None 2026-03-28 08:44:00,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:00,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:44:00,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:00,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793d6be430e18c5d160bb HTTP/1.1" 200 None 2026-03-28 08:44:00,367 - lava_callback - INFO - Completed processing callback for node 69c793d6be430e18c5d160bb INFO: 172.20.0.1:44418 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:44:07,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:07,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34224 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 08:44:16,116 - lava_callback - INFO - Received callback for node 69c793babe430e18c5d16056 INFO: 172.20.0.1:34226 - "POST /node/69c793babe430e18c5d16056 HTTP/1.0" 202 Accepted 2026-03-28 08:44:16,126 - lava_callback - INFO - Starting async job submit for node 69c793babe430e18c5d16056 2026-03-28 08:44:16,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:16,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793babe430e18c5d16056 HTTP/1.1" 200 None 2026-03-28 08:44:16,185 - lava_callback - DEBUG - Processing node 69c793babe430e18c5d16056: baseline-arm-clabbe 2026-03-28 08:44:16,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:44:16,521 - lava_callback - INFO - Received callback for node 69c793e8be430e18c5d160f2 INFO: 172.20.0.1:34240 - "POST /node/69c793e8be430e18c5d160f2 HTTP/1.0" 202 Accepted 2026-03-28 08:44:16,531 - lava_callback - INFO - Starting async job submit for node 69c793e8be430e18c5d160f2 INFO: 172.20.0.1:34252 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:44:16,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:16,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e8be430e18c5d160f2 HTTP/1.1" 200 None 2026-03-28 08:44:16,588 - lava_callback - DEBUG - Processing node 69c793e8be430e18c5d160f2: ltp-syscalls 2026-03-28 08:44:17,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:44:17,101 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793babe430e18c5d16056/log.txt.gz 2026-03-28 08:44:17,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:44:17,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:44:18,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:44:18,219 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793babe430e18c5d16056/lava-logs.yaml 2026-03-28 08:44:18,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:44:18,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:44:18,427 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c793e8be430e18c5d160f2/log.txt.gz 2026-03-28 08:44:18,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:44:18,854 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793babe430e18c5d16056/lava_callback.json.gz 2026-03-28 08:44:18,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:19,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793babe430e18c5d16056 HTTP/1.1" 200 None 2026-03-28 08:44:19,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:19,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:44:19,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:44:19,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:19,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793babe430e18c5d16056 HTTP/1.1" 200 None 2026-03-28 08:44:19,170 - lava_callback - INFO - Completed processing callback for node 69c793babe430e18c5d16056 2026-03-28 08:44:19,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:44:19,802 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c793e8be430e18c5d160f2/lava-logs.yaml 2026-03-28 08:44:19,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:44:20,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:44:20,394 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c793e8be430e18c5d160f2/lava_callback.json.gz 2026-03-28 08:44:20,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:20,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e8be430e18c5d160f2 HTTP/1.1" 200 None 2026-03-28 08:44:20,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:20,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:44:20,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:20,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793e8be430e18c5d160f2 HTTP/1.1" 200 None 2026-03-28 08:44:20,513 - lava_callback - INFO - Completed processing callback for node 69c793e8be430e18c5d160f2 INFO: 172.20.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41240 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:44:37,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:44:37,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:45:59,142 - lava_callback - INFO - Received callback for node 69c793bbbe430e18c5d16059 INFO: 172.20.0.1:56118 - "POST /node/69c793bbbe430e18c5d16059 HTTP/1.0" 202 Accepted 2026-03-28 08:45:59,148 - lava_callback - INFO - Starting async job submit for node 69c793bbbe430e18c5d16059 2026-03-28 08:45:59,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:45:59,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793bbbe430e18c5d16059 HTTP/1.1" 200 None 2026-03-28 08:45:59,187 - lava_callback - DEBUG - Processing node 69c793bbbe430e18c5d16059: baseline-arm-clabbe 2026-03-28 08:45:59,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:45:59,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:45:59,604 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793bbbe430e18c5d16059/log.txt.gz 2026-03-28 08:45:59,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:46:00,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:46:00,117 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793bbbe430e18c5d16059/lava-logs.yaml 2026-03-28 08:46:00,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:46:00,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:46:00,309 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c793bbbe430e18c5d16059/lava_callback.json.gz 2026-03-28 08:46:00,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:00,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793bbbe430e18c5d16059 HTTP/1.1" 200 None 2026-03-28 08:46:00,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:00,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:46:00,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:00,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793bbbe430e18c5d16059 HTTP/1.1" 200 None 2026-03-28 08:46:00,391 - lava_callback - INFO - Completed processing callback for node 69c793bbbe430e18c5d16059 INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:46:07,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:07,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35306 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 08:46:15,669 - lava_callback - INFO - Received callback for node 69c793d9be430e18c5d160be INFO: 172.20.0.1:35312 - "POST /node/69c793d9be430e18c5d160be HTTP/1.0" 202 Accepted 2026-03-28 08:46:15,676 - lava_callback - INFO - Starting async job submit for node 69c793d9be430e18c5d160be 2026-03-28 08:46:15,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:15,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d9be430e18c5d160be HTTP/1.1" 200 None 2026-03-28 08:46:15,737 - lava_callback - DEBUG - Processing node 69c793d9be430e18c5d160be: ltp-cve 2026-03-28 08:46:16,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:46:16,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:46:16,410 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c793d9be430e18c5d160be/log.txt.gz INFO: 172.20.0.1:35314 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:46:16,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:46:17,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:46:17,034 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c793d9be430e18c5d160be/lava-logs.yaml 2026-03-28 08:46:17,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:46:17,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:46:17,226 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c793d9be430e18c5d160be/lava_callback.json.gz 2026-03-28 08:46:17,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:17,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793d9be430e18c5d160be HTTP/1.1" 200 None 2026-03-28 08:46:17,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:17,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:46:17,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:17,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793d9be430e18c5d160be HTTP/1.1" 200 None 2026-03-28 08:46:17,319 - lava_callback - INFO - Completed processing callback for node 69c793d9be430e18c5d160be INFO: 172.20.0.1:48842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51294 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:46:37,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:46:37,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57738 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:47:27,231 - lava_callback - INFO - Received callback for node 69c7911ebe430e18c5d15fe1 INFO: 172.20.0.1:57740 - "POST /node/69c7911ebe430e18c5d15fe1 HTTP/1.0" 202 Accepted 2026-03-28 08:47:27,238 - lava_callback - INFO - Starting async job submit for node 69c7911ebe430e18c5d15fe1 2026-03-28 08:47:27,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:27,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7911ebe430e18c5d15fe1 HTTP/1.1" 200 None 2026-03-28 08:47:27,320 - lava_callback - DEBUG - Processing node 69c7911ebe430e18c5d15fe1: ltp-smoke 2026-03-28 08:47:27,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:28,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:28,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7911ebe430e18c5d15fe1/log.txt.gz 2026-03-28 08:47:28,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:29,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:29,197 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7911ebe430e18c5d15fe1/lava-logs.yaml 2026-03-28 08:47:29,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:29,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:29,816 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7911ebe430e18c5d15fe1/lava_callback.json.gz 2026-03-28 08:47:29,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:29,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7911ebe430e18c5d15fe1 HTTP/1.1" 200 None 2026-03-28 08:47:29,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:29,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb1be430e18c5d159e5 HTTP/1.1" 200 None 2026-03-28 08:47:29,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:30,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7911ebe430e18c5d15fe1 HTTP/1.1" 200 None 2026-03-28 08:47:30,165 - lava_callback - INFO - Completed processing callback for node 69c7911ebe430e18c5d15fe1 INFO: 172.20.0.1:44150 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:47:37,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:37,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59800 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:47:47,141 - lava_callback - INFO - Received callback for node 69c793dfbe430e18c5d160ca INFO: 172.20.0.1:59810 - "POST /node/69c793dfbe430e18c5d160ca HTTP/1.0" 202 Accepted 2026-03-28 08:47:47,147 - lava_callback - INFO - Starting async job submit for node 69c793dfbe430e18c5d160ca 2026-03-28 08:47:47,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:47,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793dfbe430e18c5d160ca HTTP/1.1" 200 None 2026-03-28 08:47:47,187 - lava_callback - DEBUG - Processing node 69c793dfbe430e18c5d160ca: ltp-input 2026-03-28 08:47:47,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:47,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:47,666 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c793dfbe430e18c5d160ca/log.txt.gz 2026-03-28 08:47:47,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:48,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:48,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c793dfbe430e18c5d160ca/lava-logs.yaml 2026-03-28 08:47:48,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:48,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:48,233 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c793dfbe430e18c5d160ca/lava_callback.json.gz 2026-03-28 08:47:48,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:48,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793dfbe430e18c5d160ca HTTP/1.1" 200 None 2026-03-28 08:47:48,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:48,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:47:48,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:48,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793dfbe430e18c5d160ca HTTP/1.1" 200 None 2026-03-28 08:47:48,328 - lava_callback - INFO - Completed processing callback for node 69c793dfbe430e18c5d160ca 2026-03-28 08:47:55,751 - lava_callback - INFO - Received callback for node 69c793ddbe430e18c5d160c7 INFO: 172.20.0.1:49834 - "POST /node/69c793ddbe430e18c5d160c7 HTTP/1.0" 202 Accepted 2026-03-28 08:47:55,758 - lava_callback - INFO - Starting async job submit for node 69c793ddbe430e18c5d160c7 2026-03-28 08:47:55,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:55,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793ddbe430e18c5d160c7 HTTP/1.1" 200 None 2026-03-28 08:47:55,798 - lava_callback - DEBUG - Processing node 69c793ddbe430e18c5d160c7: ltp-fsx 2026-03-28 08:47:56,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:56,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:56,256 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c793ddbe430e18c5d160c7/log.txt.gz 2026-03-28 08:47:56,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49846 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:47:56,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:56,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c793ddbe430e18c5d160c7/lava-logs.yaml 2026-03-28 08:47:56,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:56,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:56,847 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c793ddbe430e18c5d160c7/lava_callback.json.gz 2026-03-28 08:47:56,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:56,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793ddbe430e18c5d160c7 HTTP/1.1" 200 None 2026-03-28 08:47:56,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:56,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:47:56,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:56,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793ddbe430e18c5d160c7 HTTP/1.1" 200 None 2026-03-28 08:47:56,955 - lava_callback - INFO - Completed processing callback for node 69c793ddbe430e18c5d160c7 2026-03-28 08:47:57,348 - lava_callback - INFO - Received callback for node 69c793e0be430e18c5d160cd INFO: 172.20.0.1:49862 - "POST /node/69c793e0be430e18c5d160cd HTTP/1.0" 202 Accepted 2026-03-28 08:47:57,355 - lava_callback - INFO - Starting async job submit for node 69c793e0be430e18c5d160cd 2026-03-28 08:47:57,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:57,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e0be430e18c5d160cd HTTP/1.1" 200 None 2026-03-28 08:47:57,396 - lava_callback - DEBUG - Processing node 69c793e0be430e18c5d160cd: ltp-ipc 2026-03-28 08:47:57,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:57,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:57,877 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c793e0be430e18c5d160cd/log.txt.gz 2026-03-28 08:47:58,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:58,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:58,294 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c793e0be430e18c5d160cd/lava-logs.yaml 2026-03-28 08:47:58,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:47:58,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:47:58,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c793e0be430e18c5d160cd/lava_callback.json.gz 2026-03-28 08:47:58,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:58,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e0be430e18c5d160cd HTTP/1.1" 200 None 2026-03-28 08:47:58,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:58,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:47:58,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:47:58,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793e0be430e18c5d160cd HTTP/1.1" 200 None 2026-03-28 08:47:58,552 - lava_callback - INFO - Completed processing callback for node 69c793e0be430e18c5d160cd INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:48:07,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:07,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 08:48:12,418 - lava_callback - INFO - Received callback for node 69c4f2caf63ce05331e2ad36 INFO: 172.20.0.1:51446 - "POST /node/69c4f2caf63ce05331e2ad36 HTTP/1.0" 202 Accepted 2026-03-28 08:48:12,424 - lava_callback - INFO - Starting async job submit for node 69c4f2caf63ce05331e2ad36 2026-03-28 08:48:12,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:12,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f2caf63ce05331e2ad36 HTTP/1.1" 200 None 2026-03-28 08:48:12,452 - lava_callback - DEBUG - Processing node 69c4f2caf63ce05331e2ad36: baseline-arm-baylibre 2026-03-28 08:48:12,452 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f2caf63ce05331e2ad36 2026-03-28 08:48:12,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:12,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f2caf63ce05331e2ad36 HTTP/1.1" 200 None 2026-03-28 08:48:12,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:12,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:48:12,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:12,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f2caf63ce05331e2ad36 HTTP/1.1" 200 None 2026-03-28 08:48:12,530 - lava_callback - ERROR - Error processing callback for node 69c4f2caf63ce05331e2ad36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:48:52,350 - lava_callback - INFO - Received callback for node 69c4f2f0f63ce05331e2ad43 INFO: 172.20.0.1:46712 - "POST /node/69c4f2f0f63ce05331e2ad43 HTTP/1.0" 202 Accepted 2026-03-28 08:48:52,359 - lava_callback - INFO - Starting async job submit for node 69c4f2f0f63ce05331e2ad43 2026-03-28 08:48:52,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:52,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f2f0f63ce05331e2ad43 HTTP/1.1" 200 None 2026-03-28 08:48:52,380 - lava_callback - DEBUG - Processing node 69c4f2f0f63ce05331e2ad43: baseline-arm-baylibre 2026-03-28 08:48:52,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f2f0f63ce05331e2ad43 2026-03-28 08:48:52,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:52,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f2f0f63ce05331e2ad43 HTTP/1.1" 200 None 2026-03-28 08:48:52,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:52,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:48:52,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:48:52,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f2f0f63ce05331e2ad43 HTTP/1.1" 200 None 2026-03-28 08:48:52,463 - lava_callback - ERROR - Error processing callback for node 69c4f2f0f63ce05331e2ad43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:49:32,403 - lava_callback - INFO - Received callback for node 69c4f316f63ce05331e2ad67 INFO: 172.20.0.1:40542 - "POST /node/69c4f316f63ce05331e2ad67 HTTP/1.0" 202 Accepted 2026-03-28 08:49:32,412 - lava_callback - INFO - Starting async job submit for node 69c4f316f63ce05331e2ad67 2026-03-28 08:49:32,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:49:32,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f316f63ce05331e2ad67 HTTP/1.1" 200 None 2026-03-28 08:49:32,435 - lava_callback - DEBUG - Processing node 69c4f316f63ce05331e2ad67: baseline-arm-baylibre 2026-03-28 08:49:32,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f316f63ce05331e2ad67 2026-03-28 08:49:32,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:49:32,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f316f63ce05331e2ad67 HTTP/1.1" 200 None 2026-03-28 08:49:32,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:49:32,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:49:32,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:49:32,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f316f63ce05331e2ad67 HTTP/1.1" 200 None 2026-03-28 08:49:32,503 - lava_callback - ERROR - Error processing callback for node 69c4f316f63ce05331e2ad67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56536 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:49:57,494 - lava_callback - INFO - Received callback for node 69c793dcbe430e18c5d160c4 INFO: 172.20.0.1:56552 - "POST /node/69c793dcbe430e18c5d160c4 HTTP/1.0" 202 Accepted 2026-03-28 08:49:57,504 - lava_callback - INFO - Starting async job submit for node 69c793dcbe430e18c5d160c4 2026-03-28 08:49:57,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:49:57,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793dcbe430e18c5d160c4 HTTP/1.1" 200 None 2026-03-28 08:49:57,586 - lava_callback - DEBUG - Processing node 69c793dcbe430e18c5d160c4: ltp-fs_bind 2026-03-28 08:49:58,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:49:59,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:49:59,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c793dcbe430e18c5d160c4/log.txt.gz 2026-03-28 08:49:59,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:50:00,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:50:00,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c793dcbe430e18c5d160c4/lava-logs.yaml 2026-03-28 08:50:00,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:50:01,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:50:01,976 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c793dcbe430e18c5d160c4/lava_callback.json.gz 2026-03-28 08:50:02,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:02,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793dcbe430e18c5d160c4 HTTP/1.1" 200 None 2026-03-28 08:50:02,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:02,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:50:02,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:02,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793dcbe430e18c5d160c4 HTTP/1.1" 200 None 2026-03-28 08:50:02,099 - lava_callback - INFO - Completed processing callback for node 69c793dcbe430e18c5d160c4 INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:50:07,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:07,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 08:50:12,374 - lava_callback - INFO - Received callback for node 69c4f33cf63ce05331e2ad80 INFO: 172.20.0.1:58224 - "POST /node/69c4f33cf63ce05331e2ad80 HTTP/1.0" 202 Accepted 2026-03-28 08:50:12,383 - lava_callback - INFO - Starting async job submit for node 69c4f33cf63ce05331e2ad80 2026-03-28 08:50:12,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:12,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f33cf63ce05331e2ad80 HTTP/1.1" 200 None 2026-03-28 08:50:12,410 - lava_callback - DEBUG - Processing node 69c4f33cf63ce05331e2ad80: baseline-arm-baylibre 2026-03-28 08:50:12,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f33cf63ce05331e2ad80 2026-03-28 08:50:12,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:12,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f33cf63ce05331e2ad80 HTTP/1.1" 200 None 2026-03-28 08:50:12,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:12,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:50:12,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:12,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f33cf63ce05331e2ad80 HTTP/1.1" 200 None 2026-03-28 08:50:12,503 - lava_callback - ERROR - Error processing callback for node 69c4f33cf63ce05331e2ad80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47924 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:50:44,814 - lava_callback - INFO - Received callback for node 69c79657be430e18c5d162e0 INFO: 172.20.0.1:47926 - "POST /node/69c79657be430e18c5d162e0 HTTP/1.0" 202 Accepted 2026-03-28 08:50:44,823 - lava_callback - INFO - Starting async job submit for node 69c79657be430e18c5d162e0 2026-03-28 08:50:44,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:44,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79657be430e18c5d162e0 HTTP/1.1" 200 None 2026-03-28 08:50:44,857 - lava_callback - DEBUG - Processing node 69c79657be430e18c5d162e0: baseline-arm 2026-03-28 08:50:44,865 - lava_callback - WARNING - Failed to upload log 2026-03-28 08:50:44,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:50:46,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:50:46,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c79657be430e18c5d162e0/lava-logs.yaml 2026-03-28 08:50:46,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47940 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:50:47,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:50:47,218 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c79657be430e18c5d162e0/lava_callback.json.gz 2026-03-28 08:50:47,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:47,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79657be430e18c5d162e0 HTTP/1.1" 200 None 2026-03-28 08:50:47,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:47,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:50:47,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:47,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79657be430e18c5d162e0 HTTP/1.1" 200 None 2026-03-28 08:50:47,504 - lava_callback - INFO - Completed processing callback for node 69c79657be430e18c5d162e0 2026-03-28 08:50:52,467 - lava_callback - INFO - Received callback for node 69c4f363f63ce05331e2ad90 INFO: 172.20.0.1:49846 - "POST /node/69c4f363f63ce05331e2ad90 HTTP/1.0" 202 Accepted 2026-03-28 08:50:52,473 - lava_callback - INFO - Starting async job submit for node 69c4f363f63ce05331e2ad90 2026-03-28 08:50:52,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:52,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f363f63ce05331e2ad90 HTTP/1.1" 200 None 2026-03-28 08:50:52,513 - lava_callback - DEBUG - Processing node 69c4f363f63ce05331e2ad90: baseline-arm-baylibre 2026-03-28 08:50:52,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f363f63ce05331e2ad90 2026-03-28 08:50:52,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:52,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f363f63ce05331e2ad90 HTTP/1.1" 200 None 2026-03-28 08:50:52,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:52,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:50:52,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:52,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f363f63ce05331e2ad90 HTTP/1.1" 200 None 2026-03-28 08:50:52,589 - lava_callback - ERROR - Error processing callback for node 69c4f363f63ce05331e2ad90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:50:55,058 - lava_callback - INFO - Received callback for node 69c79652be430e18c5d162d8 INFO: 172.20.0.1:49852 - "POST /node/69c79652be430e18c5d162d8 HTTP/1.0" 202 Accepted 2026-03-28 08:50:55,064 - lava_callback - INFO - Starting async job submit for node 69c79652be430e18c5d162d8 2026-03-28 08:50:55,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:55,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79652be430e18c5d162d8 HTTP/1.1" 200 None 2026-03-28 08:50:55,116 - lava_callback - DEBUG - Processing node 69c79652be430e18c5d162d8: baseline-x86 2026-03-28 08:50:55,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:50:56,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:50:56,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c79652be430e18c5d162d8/log.txt.gz INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:50:56,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:50:57,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:50:57,791 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c79652be430e18c5d162d8/lava-logs.yaml 2026-03-28 08:50:57,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:50:58,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:50:58,878 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c79652be430e18c5d162d8/lava_callback.json.gz 2026-03-28 08:50:58,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:58,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79652be430e18c5d162d8 HTTP/1.1" 200 None 2026-03-28 08:50:58,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:58,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 08:50:58,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:59,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79652be430e18c5d162d8 HTTP/1.1" 200 None 2026-03-28 08:50:59,007 - lava_callback - INFO - Completed processing callback for node 69c79652be430e18c5d162d8 2026-03-28 08:50:59,034 - lava_callback - INFO - Received callback for node 69c7964fbe430e18c5d162cf INFO: 172.20.0.1:49880 - "POST /node/69c7964fbe430e18c5d162cf HTTP/1.0" 202 Accepted 2026-03-28 08:50:59,056 - lava_callback - INFO - Starting async job submit for node 69c7964fbe430e18c5d162cf 2026-03-28 08:50:59,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:50:59,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7964fbe430e18c5d162cf HTTP/1.1" 200 None 2026-03-28 08:50:59,145 - lava_callback - DEBUG - Processing node 69c7964fbe430e18c5d162cf: baseline-x86 2026-03-28 08:50:59,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:00,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:00,616 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c7964fbe430e18c5d162cf/log.txt.gz 2026-03-28 08:51:00,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:01,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:01,960 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c7964fbe430e18c5d162cf/lava-logs.yaml 2026-03-28 08:51:01,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:03,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:03,024 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c7964fbe430e18c5d162cf/lava_callback.json.gz 2026-03-28 08:51:03,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:03,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7964fbe430e18c5d162cf HTTP/1.1" 200 None 2026-03-28 08:51:03,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:03,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 08:51:03,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:03,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7964fbe430e18c5d162cf HTTP/1.1" 200 None 2026-03-28 08:51:03,137 - lava_callback - INFO - Completed processing callback for node 69c7964fbe430e18c5d162cf INFO: 172.20.0.1:57630 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:51:07,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:07,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:51:32,413 - lava_callback - INFO - Received callback for node 69c4f389f63ce05331e2ada2 INFO: 172.20.0.1:44748 - "POST /node/69c4f389f63ce05331e2ada2 HTTP/1.0" 202 Accepted 2026-03-28 08:51:32,420 - lava_callback - INFO - Starting async job submit for node 69c4f389f63ce05331e2ada2 2026-03-28 08:51:32,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:32,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f389f63ce05331e2ada2 HTTP/1.1" 200 None 2026-03-28 08:51:32,451 - lava_callback - DEBUG - Processing node 69c4f389f63ce05331e2ada2: baseline-arm-baylibre 2026-03-28 08:51:32,452 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f389f63ce05331e2ada2 2026-03-28 08:51:32,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:32,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f389f63ce05331e2ada2 HTTP/1.1" 200 None 2026-03-28 08:51:32,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:32,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:51:32,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:32,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f389f63ce05331e2ada2 HTTP/1.1" 200 None 2026-03-28 08:51:32,522 - lava_callback - ERROR - Error processing callback for node 69c4f389f63ce05331e2ada2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:51:35,831 - lava_callback - INFO - Received callback for node 69c793e3be430e18c5d160d0 INFO: 172.20.0.1:44752 - "POST /node/69c793e3be430e18c5d160d0 HTTP/1.0" 202 Accepted 2026-03-28 08:51:35,839 - lava_callback - INFO - Starting async job submit for node 69c793e3be430e18c5d160d0 2026-03-28 08:51:35,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:35,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e3be430e18c5d160d0 HTTP/1.1" 200 None 2026-03-28 08:51:35,888 - lava_callback - DEBUG - Processing node 69c793e3be430e18c5d160d0: ltp-pty 2026-03-28 08:51:36,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:44760 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:51:37,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:37,291 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c793e3be430e18c5d160d0/log.txt.gz 2026-03-28 08:51:37,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:38,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:38,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c793e3be430e18c5d160d0/lava-logs.yaml 2026-03-28 08:51:38,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:39,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:39,622 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c793e3be430e18c5d160d0/lava_callback.json.gz 2026-03-28 08:51:39,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:39,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e3be430e18c5d160d0 HTTP/1.1" 200 None 2026-03-28 08:51:39,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:39,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:51:39,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:39,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793e3be430e18c5d160d0 HTTP/1.1" 200 None 2026-03-28 08:51:39,753 - lava_callback - INFO - Completed processing callback for node 69c793e3be430e18c5d160d0 INFO: 172.20.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57026 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:51:56,139 - lava_callback - INFO - Received callback for node 69c793e4be430e18c5d160d3 INFO: 172.20.0.1:54562 - "POST /node/69c793e4be430e18c5d160d3 HTTP/1.0" 202 Accepted 2026-03-28 08:51:56,146 - lava_callback - INFO - Starting async job submit for node 69c793e4be430e18c5d160d3 2026-03-28 08:51:56,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:56,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e4be430e18c5d160d3 HTTP/1.1" 200 None 2026-03-28 08:51:56,191 - lava_callback - DEBUG - Processing node 69c793e4be430e18c5d160d3: ltp-sched INFO: 172.20.0.1:54578 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:51:56,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:56,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:56,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c793e4be430e18c5d160d3/log.txt.gz 2026-03-28 08:51:57,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:57,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:57,308 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c793e4be430e18c5d160d3/lava-logs.yaml 2026-03-28 08:51:57,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:57,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:57,507 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c793e4be430e18c5d160d3/lava_callback.json.gz 2026-03-28 08:51:57,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:57,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e4be430e18c5d160d3 HTTP/1.1" 200 None 2026-03-28 08:51:57,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:57,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:51:57,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:57,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793e4be430e18c5d160d3 HTTP/1.1" 200 None 2026-03-28 08:51:57,626 - lava_callback - INFO - Completed processing callback for node 69c793e4be430e18c5d160d3 2026-03-28 08:51:58,045 - lava_callback - INFO - Received callback for node 69c793e6be430e18c5d160ec INFO: 172.20.0.1:54592 - "POST /node/69c793e6be430e18c5d160ec HTTP/1.0" 202 Accepted 2026-03-28 08:51:58,055 - lava_callback - INFO - Starting async job submit for node 69c793e6be430e18c5d160ec 2026-03-28 08:51:58,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:58,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e6be430e18c5d160ec HTTP/1.1" 200 None 2026-03-28 08:51:58,130 - lava_callback - DEBUG - Processing node 69c793e6be430e18c5d160ec: ltp-smoke 2026-03-28 08:51:58,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:58,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:58,701 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c793e6be430e18c5d160ec/log.txt.gz 2026-03-28 08:51:58,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:59,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:59,249 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c793e6be430e18c5d160ec/lava-logs.yaml 2026-03-28 08:51:59,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:51:59,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:51:59,442 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c793e6be430e18c5d160ec/lava_callback.json.gz 2026-03-28 08:51:59,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:59,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e6be430e18c5d160ec HTTP/1.1" 200 None 2026-03-28 08:51:59,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:59,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:51:59,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:51:59,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793e6be430e18c5d160ec HTTP/1.1" 200 None 2026-03-28 08:51:59,541 - lava_callback - INFO - Completed processing callback for node 69c793e6be430e18c5d160ec 2026-03-28 08:52:03,545 - lava_callback - INFO - Received callback for node 69c4f3b0f63ce05331e2adad INFO: 172.20.0.1:59396 - "POST /node/69c4f3b0f63ce05331e2adad HTTP/1.0" 202 Accepted 2026-03-28 08:52:03,554 - lava_callback - INFO - Starting async job submit for node 69c4f3b0f63ce05331e2adad 2026-03-28 08:52:03,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:03,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f3b0f63ce05331e2adad HTTP/1.1" 200 None 2026-03-28 08:52:03,576 - lava_callback - DEBUG - Processing node 69c4f3b0f63ce05331e2adad: baseline-arm-clabbe 2026-03-28 08:52:03,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f3b0f63ce05331e2adad 2026-03-28 08:52:03,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:03,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f3b0f63ce05331e2adad HTTP/1.1" 200 None 2026-03-28 08:52:03,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:03,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:52:03,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:03,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f3b0f63ce05331e2adad HTTP/1.1" 200 None 2026-03-28 08:52:03,652 - lava_callback - ERROR - Error processing callback for node 69c4f3b0f63ce05331e2adad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59404 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:52:06,946 - lava_callback - INFO - Received callback for node 69c7963bbe430e18c5d162bf INFO: 172.20.0.1:59408 - "POST /node/69c7963bbe430e18c5d162bf HTTP/1.0" 202 Accepted 2026-03-28 08:52:06,956 - lava_callback - INFO - Starting async job submit for node 69c7963bbe430e18c5d162bf 2026-03-28 08:52:06,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:07,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7963bbe430e18c5d162bf HTTP/1.1" 200 None 2026-03-28 08:52:07,025 - lava_callback - DEBUG - Processing node 69c7963bbe430e18c5d162bf: baseline-arm-pengutronix 2026-03-28 08:52:07,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:52:07,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:52:07,435 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7963bbe430e18c5d162bf/log.txt.gz 2026-03-28 08:52:07,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:52:07,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:52:07,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7963bbe430e18c5d162bf/lava-logs.yaml 2026-03-28 08:52:07,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:52:07,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:07,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 08:52:07,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:52:07,881 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7963bbe430e18c5d162bf/lava_callback.json.gz 2026-03-28 08:52:07,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:07,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7963bbe430e18c5d162bf HTTP/1.1" 200 None 2026-03-28 08:52:07,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:07,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:52:07,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:08,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7963bbe430e18c5d162bf HTTP/1.1" 200 None 2026-03-28 08:52:08,124 - lava_callback - INFO - Completed processing callback for node 69c7963bbe430e18c5d162bf 2026-03-28 08:52:14,155 - lava_callback - INFO - Received callback for node 69c79650be430e18c5d162d2 INFO: 172.20.0.1:48430 - "POST /node/69c79650be430e18c5d162d2 HTTP/1.0" 202 Accepted 2026-03-28 08:52:14,162 - lava_callback - INFO - Starting async job submit for node 69c79650be430e18c5d162d2 2026-03-28 08:52:14,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:14,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79650be430e18c5d162d2 HTTP/1.1" 200 None 2026-03-28 08:52:14,208 - lava_callback - DEBUG - Processing node 69c79650be430e18c5d162d2: baseline-x86 2026-03-28 08:52:14,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:52:14,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:52:14,666 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c79650be430e18c5d162d2/log.txt.gz 2026-03-28 08:52:14,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48438 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 08:52:15,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:52:15,186 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c79650be430e18c5d162d2/lava-logs.yaml 2026-03-28 08:52:15,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:52:15,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:52:15,325 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c79650be430e18c5d162d2/lava_callback.json.gz 2026-03-28 08:52:15,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:15,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79650be430e18c5d162d2 HTTP/1.1" 200 None 2026-03-28 08:52:15,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:15,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 08:52:15,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:15,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79650be430e18c5d162d2 HTTP/1.1" 200 None 2026-03-28 08:52:15,407 - lava_callback - INFO - Completed processing callback for node 69c79650be430e18c5d162d2 INFO: 172.20.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:52:37,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:37,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52798 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:52:43,489 - lava_callback - INFO - Received callback for node 69c4f3d6f63ce05331e2adbb INFO: 172.20.0.1:52800 - "POST /node/69c4f3d6f63ce05331e2adbb HTTP/1.0" 202 Accepted 2026-03-28 08:52:43,495 - lava_callback - INFO - Starting async job submit for node 69c4f3d6f63ce05331e2adbb 2026-03-28 08:52:43,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:43,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f3d6f63ce05331e2adbb HTTP/1.1" 200 None 2026-03-28 08:52:43,517 - lava_callback - DEBUG - Processing node 69c4f3d6f63ce05331e2adbb: baseline-arm-clabbe 2026-03-28 08:52:43,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f3d6f63ce05331e2adbb 2026-03-28 08:52:43,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:43,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f3d6f63ce05331e2adbb HTTP/1.1" 200 None 2026-03-28 08:52:43,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:43,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:52:43,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:52:43,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f3d6f63ce05331e2adbb HTTP/1.1" 200 None 2026-03-28 08:52:43,602 - lava_callback - ERROR - Error processing callback for node 69c4f3d6f63ce05331e2adbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:53:23,544 - lava_callback - INFO - Received callback for node 69c4f3fbf63ce05331e2ade0 INFO: 172.20.0.1:35142 - "POST /node/69c4f3fbf63ce05331e2ade0 HTTP/1.0" 202 Accepted 2026-03-28 08:53:23,551 - lava_callback - INFO - Starting async job submit for node 69c4f3fbf63ce05331e2ade0 2026-03-28 08:53:23,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:53:23,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f3fbf63ce05331e2ade0 HTTP/1.1" 200 None 2026-03-28 08:53:23,570 - lava_callback - DEBUG - Processing node 69c4f3fbf63ce05331e2ade0: baseline-arm-clabbe 2026-03-28 08:53:23,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f3fbf63ce05331e2ade0 2026-03-28 08:53:23,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:53:23,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f3fbf63ce05331e2ade0 HTTP/1.1" 200 None 2026-03-28 08:53:23,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:53:23,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:53:23,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:53:23,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f3fbf63ce05331e2ade0 HTTP/1.1" 200 None 2026-03-28 08:53:23,634 - lava_callback - ERROR - Error processing callback for node 69c4f3fbf63ce05331e2ade0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51418 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:54:03,554 - lava_callback - INFO - Received callback for node 69c4f420f63ce05331e2adf6 INFO: 172.20.0.1:33488 - "POST /node/69c4f420f63ce05331e2adf6 HTTP/1.0" 202 Accepted 2026-03-28 08:54:03,563 - lava_callback - INFO - Starting async job submit for node 69c4f420f63ce05331e2adf6 2026-03-28 08:54:03,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:03,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f420f63ce05331e2adf6 HTTP/1.1" 200 None 2026-03-28 08:54:03,613 - lava_callback - DEBUG - Processing node 69c4f420f63ce05331e2adf6: baseline-arm-clabbe 2026-03-28 08:54:03,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f420f63ce05331e2adf6 2026-03-28 08:54:03,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:03,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f420f63ce05331e2adf6 HTTP/1.1" 200 None 2026-03-28 08:54:03,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:03,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:54:03,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:03,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f420f63ce05331e2adf6 HTTP/1.1" 200 None 2026-03-28 08:54:03,693 - lava_callback - ERROR - Error processing callback for node 69c4f420f63ce05331e2adf6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33496 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:54:11,449 - lava_callback - INFO - Received callback for node 69c79715be430e18c5d16384 INFO: 172.20.0.1:58110 - "POST /node/69c79715be430e18c5d16384 HTTP/1.0" 202 Accepted 2026-03-28 08:54:11,455 - lava_callback - INFO - Starting async job submit for node 69c79715be430e18c5d16384 2026-03-28 08:54:11,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:11,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79715be430e18c5d16384 HTTP/1.1" 200 None 2026-03-28 08:54:11,490 - lava_callback - DEBUG - Processing node 69c79715be430e18c5d16384: baseline-arm64 2026-03-28 08:54:11,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:11,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:11,848 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79715be430e18c5d16384/log.txt.gz 2026-03-28 08:54:11,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:12,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:12,118 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79715be430e18c5d16384/lava-logs.yaml 2026-03-28 08:54:12,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:12,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:12,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79715be430e18c5d16384/lava_callback.json.gz 2026-03-28 08:54:12,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:12,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79715be430e18c5d16384 HTTP/1.1" 200 None 2026-03-28 08:54:12,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:12,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:12,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:12,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79715be430e18c5d16384 HTTP/1.1" 200 None 2026-03-28 08:54:12,448 - lava_callback - INFO - Completed processing callback for node 69c79715be430e18c5d16384 INFO: 172.20.0.1:58114 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 08:54:16,016 - lava_callback - INFO - Received callback for node 69c793ebbe430e18c5d160f8 INFO: 172.20.0.1:58128 - "POST /node/69c793ebbe430e18c5d160f8 HTTP/1.0" 202 Accepted 2026-03-28 08:54:16,023 - lava_callback - INFO - Starting async job submit for node 69c793ebbe430e18c5d160f8 2026-03-28 08:54:16,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:16,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793ebbe430e18c5d160f8 HTTP/1.1" 200 None 2026-03-28 08:54:16,065 - lava_callback - DEBUG - Processing node 69c793ebbe430e18c5d160f8: ltp-timers INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:54:16,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:16,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:16,695 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c793ebbe430e18c5d160f8/log.txt.gz 2026-03-28 08:54:16,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:17,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:17,160 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c793ebbe430e18c5d160f8/lava-logs.yaml 2026-03-28 08:54:17,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:17,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:17,338 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c793ebbe430e18c5d160f8/lava_callback.json.gz 2026-03-28 08:54:17,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:17,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793ebbe430e18c5d160f8 HTTP/1.1" 200 None 2026-03-28 08:54:17,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:17,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:54:17,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:17,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793ebbe430e18c5d160f8 HTTP/1.1" 200 None 2026-03-28 08:54:17,425 - lava_callback - INFO - Completed processing callback for node 69c793ebbe430e18c5d160f8 2026-03-28 08:54:17,480 - lava_callback - INFO - Received callback for node 69c79732be430e18c5d163ed INFO: 172.20.0.1:58134 - "POST /node/69c79732be430e18c5d163ed HTTP/1.0" 202 Accepted 2026-03-28 08:54:17,490 - lava_callback - INFO - Starting async job submit for node 69c79732be430e18c5d163ed 2026-03-28 08:54:17,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:17,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79732be430e18c5d163ed HTTP/1.1" 200 None 2026-03-28 08:54:17,516 - lava_callback - DEBUG - Processing node 69c79732be430e18c5d163ed: baseline-nfs-arm64 2026-03-28 08:54:17,518 - lava_callback - WARNING - Failed to upload log 2026-03-28 08:54:17,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:17,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:17,692 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79732be430e18c5d163ed/lava-logs.yaml 2026-03-28 08:54:17,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:17,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:17,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79732be430e18c5d163ed/lava_callback.json.gz 2026-03-28 08:54:17,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:17,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79732be430e18c5d163ed HTTP/1.1" 200 None 2026-03-28 08:54:17,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:17,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:17,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:17,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79732be430e18c5d163ed HTTP/1.1" 200 None 2026-03-28 08:54:17,952 - lava_callback - INFO - Completed processing callback for node 69c79732be430e18c5d163ed INFO: 172.20.0.1:45290 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:54:33,837 - lava_callback - INFO - Received callback for node 69c79722be430e18c5d163a5 INFO: 172.20.0.1:47898 - "POST /node/69c79722be430e18c5d163a5 HTTP/1.0" 202 Accepted 2026-03-28 08:54:33,844 - lava_callback - INFO - Starting async job submit for node 69c79722be430e18c5d163a5 2026-03-28 08:54:33,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:33,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79722be430e18c5d163a5 HTTP/1.1" 200 None 2026-03-28 08:54:33,877 - lava_callback - DEBUG - Processing node 69c79722be430e18c5d163a5: baseline-arm64 2026-03-28 08:54:34,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:34,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:34,145 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79722be430e18c5d163a5/log.txt.gz 2026-03-28 08:54:34,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:34,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:34,528 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79722be430e18c5d163a5/lava-logs.yaml 2026-03-28 08:54:34,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:34,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:34,677 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79722be430e18c5d163a5/lava_callback.json.gz 2026-03-28 08:54:34,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:34,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79722be430e18c5d163a5 HTTP/1.1" 200 None 2026-03-28 08:54:34,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:34,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:34,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:34,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79722be430e18c5d163a5 HTTP/1.1" 200 None 2026-03-28 08:54:34,757 - lava_callback - INFO - Completed processing callback for node 69c79722be430e18c5d163a5 INFO: 172.20.0.1:47904 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:54:37,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:37,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-28 08:54:40,641 - lava_callback - INFO - Received callback for node 69c7971ebe430e18c5d1639c INFO: 172.20.0.1:54280 - "POST /node/69c7971ebe430e18c5d1639c HTTP/1.0" 202 Accepted 2026-03-28 08:54:40,655 - lava_callback - INFO - Starting async job submit for node 69c7971ebe430e18c5d1639c 2026-03-28 08:54:40,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:40,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971ebe430e18c5d1639c HTTP/1.1" 200 None 2026-03-28 08:54:40,719 - lava_callback - DEBUG - Processing node 69c7971ebe430e18c5d1639c: baseline-arm64 2026-03-28 08:54:41,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:41,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:41,261 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971ebe430e18c5d1639c/log.txt.gz 2026-03-28 08:54:41,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:41,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:41,690 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971ebe430e18c5d1639c/lava-logs.yaml 2026-03-28 08:54:41,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:41,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:41,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971ebe430e18c5d1639c/lava_callback.json.gz 2026-03-28 08:54:41,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:41,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971ebe430e18c5d1639c HTTP/1.1" 200 None 2026-03-28 08:54:41,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:41,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:41,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:41,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7971ebe430e18c5d1639c HTTP/1.1" 200 None 2026-03-28 08:54:41,917 - lava_callback - INFO - Completed processing callback for node 69c7971ebe430e18c5d1639c INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:54:43,607 - lava_callback - INFO - Received callback for node 69c4f446f63ce05331e2ae01 INFO: 172.20.0.1:54304 - "POST /node/69c4f446f63ce05331e2ae01 HTTP/1.0" 202 Accepted 2026-03-28 08:54:43,612 - lava_callback - INFO - Starting async job submit for node 69c4f446f63ce05331e2ae01 2026-03-28 08:54:43,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:43,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f446f63ce05331e2ae01 HTTP/1.1" 200 None 2026-03-28 08:54:43,629 - lava_callback - DEBUG - Processing node 69c4f446f63ce05331e2ae01: baseline-arm-clabbe 2026-03-28 08:54:43,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f446f63ce05331e2ae01 2026-03-28 08:54:43,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:43,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f446f63ce05331e2ae01 HTTP/1.1" 200 None 2026-03-28 08:54:43,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:43,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 08:54:43,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:43,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f446f63ce05331e2ae01 HTTP/1.1" 200 None 2026-03-28 08:54:43,701 - lava_callback - ERROR - Error processing callback for node 69c4f446f63ce05331e2ae01: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54318 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:54:50,403 - lava_callback - INFO - Received callback for node 69c79719be430e18c5d16390 INFO: 172.20.0.1:40452 - "POST /node/69c79719be430e18c5d16390 HTTP/1.0" 202 Accepted 2026-03-28 08:54:50,411 - lava_callback - INFO - Starting async job submit for node 69c79719be430e18c5d16390 2026-03-28 08:54:50,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:50,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79719be430e18c5d16390 HTTP/1.1" 200 None 2026-03-28 08:54:50,458 - lava_callback - DEBUG - Processing node 69c79719be430e18c5d16390: baseline-arm64 2026-03-28 08:54:50,591 - lava_callback - WARNING - Failed to upload log 2026-03-28 08:54:50,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:50,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:50,809 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79719be430e18c5d16390/lava-logs.yaml 2026-03-28 08:54:50,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:50,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:50,940 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79719be430e18c5d16390/lava_callback.json.gz 2026-03-28 08:54:51,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:51,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79719be430e18c5d16390 HTTP/1.1" 200 None 2026-03-28 08:54:51,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:51,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:51,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:51,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79719be430e18c5d16390 HTTP/1.1" 200 None 2026-03-28 08:54:51,158 - lava_callback - INFO - Completed processing callback for node 69c79719be430e18c5d16390 2026-03-28 08:54:51,514 - lava_callback - INFO - Received callback for node 69c7971bbe430e18c5d16393 INFO: 172.20.0.1:40464 - "POST /node/69c7971bbe430e18c5d16393 HTTP/1.0" 202 Accepted 2026-03-28 08:54:51,521 - lava_callback - INFO - Starting async job submit for node 69c7971bbe430e18c5d16393 2026-03-28 08:54:51,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:51,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971bbe430e18c5d16393 HTTP/1.1" 200 None 2026-03-28 08:54:51,573 - lava_callback - DEBUG - Processing node 69c7971bbe430e18c5d16393: baseline-arm64 2026-03-28 08:54:51,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:51,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:51,842 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971bbe430e18c5d16393/log.txt.gz 2026-03-28 08:54:51,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:52,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:52,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971bbe430e18c5d16393/lava-logs.yaml 2026-03-28 08:54:52,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:52,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:52,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971bbe430e18c5d16393/lava_callback.json.gz 2026-03-28 08:54:52,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:52,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971bbe430e18c5d16393 HTTP/1.1" 200 None 2026-03-28 08:54:52,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:52,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:52,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:52,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7971bbe430e18c5d16393 HTTP/1.1" 200 None 2026-03-28 08:54:52,439 - lava_callback - INFO - Completed processing callback for node 69c7971bbe430e18c5d16393 2026-03-28 08:54:53,696 - lava_callback - INFO - Received callback for node 69c79717be430e18c5d1638a INFO: 172.20.0.1:40474 - "POST /node/69c79717be430e18c5d1638a HTTP/1.0" 202 Accepted 2026-03-28 08:54:53,703 - lava_callback - INFO - Starting async job submit for node 69c79717be430e18c5d1638a 2026-03-28 08:54:53,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:53,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79717be430e18c5d1638a HTTP/1.1" 200 None 2026-03-28 08:54:53,754 - lava_callback - DEBUG - Processing node 69c79717be430e18c5d1638a: baseline-arm64 2026-03-28 08:54:54,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:54,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:54,486 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79717be430e18c5d1638a/log.txt.gz 2026-03-28 08:54:54,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:54,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:54,924 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79717be430e18c5d1638a/lava-logs.yaml 2026-03-28 08:54:54,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:55,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:55,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79717be430e18c5d1638a/lava_callback.json.gz 2026-03-28 08:54:55,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:55,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79717be430e18c5d1638a HTTP/1.1" 200 None 2026-03-28 08:54:55,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:55,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:55,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:55,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79717be430e18c5d1638a HTTP/1.1" 200 None 2026-03-28 08:54:55,151 - lava_callback - INFO - Completed processing callback for node 69c79717be430e18c5d1638a 2026-03-28 08:54:55,339 - lava_callback - INFO - Received callback for node 69c79723be430e18c5d163a8 INFO: 172.20.0.1:40486 - "POST /node/69c79723be430e18c5d163a8 HTTP/1.0" 202 Accepted 2026-03-28 08:54:55,354 - lava_callback - INFO - Starting async job submit for node 69c79723be430e18c5d163a8 2026-03-28 08:54:55,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:55,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79723be430e18c5d163a8 HTTP/1.1" 200 None 2026-03-28 08:54:55,425 - lava_callback - DEBUG - Processing node 69c79723be430e18c5d163a8: baseline-arm64 2026-03-28 08:54:55,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:55,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:55,842 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79723be430e18c5d163a8/log.txt.gz 2026-03-28 08:54:55,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:56,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:56,186 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79723be430e18c5d163a8/lava-logs.yaml 2026-03-28 08:54:56,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:54:56,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:54:56,370 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79723be430e18c5d163a8/lava_callback.json.gz 2026-03-28 08:54:56,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:56,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79723be430e18c5d163a8 HTTP/1.1" 200 None 2026-03-28 08:54:56,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:56,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:54:56,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:54:56,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79723be430e18c5d163a8 HTTP/1.1" 200 None 2026-03-28 08:54:56,451 - lava_callback - INFO - Completed processing callback for node 69c79723be430e18c5d163a8 INFO: 172.20.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51794 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:55:07,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:07,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-28 08:55:08,101 - lava_callback - INFO - Received callback for node 69c7971dbe430e18c5d16399 INFO: 172.20.0.1:51804 - "POST /node/69c7971dbe430e18c5d16399 HTTP/1.0" 202 Accepted 2026-03-28 08:55:08,107 - lava_callback - INFO - Starting async job submit for node 69c7971dbe430e18c5d16399 2026-03-28 08:55:08,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:08,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971dbe430e18c5d16399 HTTP/1.1" 200 None 2026-03-28 08:55:08,145 - lava_callback - DEBUG - Processing node 69c7971dbe430e18c5d16399: baseline-arm64 2026-03-28 08:55:08,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:08,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:08,531 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971dbe430e18c5d16399/log.txt.gz 2026-03-28 08:55:08,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:08,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:08,886 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971dbe430e18c5d16399/lava-logs.yaml 2026-03-28 08:55:08,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:09,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:09,025 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971dbe430e18c5d16399/lava_callback.json.gz 2026-03-28 08:55:09,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:09,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971dbe430e18c5d16399 HTTP/1.1" 200 None 2026-03-28 08:55:09,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:09,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:09,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:09,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7971dbe430e18c5d16399 HTTP/1.1" 200 None 2026-03-28 08:55:09,116 - lava_callback - INFO - Completed processing callback for node 69c7971dbe430e18c5d16399 INFO: 172.20.0.1:47340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47356 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:55:20,344 - lava_callback - INFO - Received callback for node 69c79716be430e18c5d16387 INFO: 172.20.0.1:54138 - "POST /node/69c79716be430e18c5d16387 HTTP/1.0" 202 Accepted 2026-03-28 08:55:20,351 - lava_callback - INFO - Starting async job submit for node 69c79716be430e18c5d16387 2026-03-28 08:55:20,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:20,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79716be430e18c5d16387 HTTP/1.1" 200 None 2026-03-28 08:55:20,388 - lava_callback - DEBUG - Processing node 69c79716be430e18c5d16387: baseline-arm64 2026-03-28 08:55:20,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:20,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:20,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79716be430e18c5d16387/log.txt.gz 2026-03-28 08:55:20,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:20,899 - lava_callback - INFO - Received callback for node 69c79721be430e18c5d163a2 INFO: 172.20.0.1:54150 - "POST /node/69c79721be430e18c5d163a2 HTTP/1.0" 202 Accepted 2026-03-28 08:55:20,910 - lava_callback - INFO - Starting async job submit for node 69c79721be430e18c5d163a2 2026-03-28 08:55:20,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:20,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79721be430e18c5d163a2 HTTP/1.1" 200 None 2026-03-28 08:55:20,968 - lava_callback - DEBUG - Processing node 69c79721be430e18c5d163a2: baseline-arm64 2026-03-28 08:55:21,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:21,154 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79716be430e18c5d16387/lava-logs.yaml 2026-03-28 08:55:21,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:21,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:21,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:21,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79716be430e18c5d16387/lava_callback.json.gz 2026-03-28 08:55:21,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:21,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:21,508 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79721be430e18c5d163a2/log.txt.gz 2026-03-28 08:55:21,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79716be430e18c5d16387 HTTP/1.1" 200 None 2026-03-28 08:55:21,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:21,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:21,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:21,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:21,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79716be430e18c5d16387 HTTP/1.1" 200 None 2026-03-28 08:55:21,799 - lava_callback - INFO - Completed processing callback for node 69c79716be430e18c5d16387 2026-03-28 08:55:22,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:22,028 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79721be430e18c5d163a2/lava-logs.yaml 2026-03-28 08:55:22,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:22,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:22,210 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79721be430e18c5d163a2/lava_callback.json.gz 2026-03-28 08:55:22,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:22,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79721be430e18c5d163a2 HTTP/1.1" 200 None 2026-03-28 08:55:22,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:22,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:22,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:22,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79721be430e18c5d163a2 HTTP/1.1" 200 None 2026-03-28 08:55:22,400 - lava_callback - INFO - Completed processing callback for node 69c79721be430e18c5d163a2 2026-03-28 08:55:25,313 - lava_callback - INFO - Received callback for node 69c7972fbe430e18c5d163e7 INFO: 172.20.0.1:54156 - "POST /node/69c7972fbe430e18c5d163e7 HTTP/1.0" 202 Accepted 2026-03-28 08:55:25,320 - lava_callback - INFO - Starting async job submit for node 69c7972fbe430e18c5d163e7 2026-03-28 08:55:25,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:25,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972fbe430e18c5d163e7 HTTP/1.1" 200 None 2026-03-28 08:55:25,357 - lava_callback - DEBUG - Processing node 69c7972fbe430e18c5d163e7: baseline-nfs-arm64 2026-03-28 08:55:25,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:25,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:25,955 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972fbe430e18c5d163e7/log.txt.gz 2026-03-28 08:55:26,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:26,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:26,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972fbe430e18c5d163e7/lava-logs.yaml 2026-03-28 08:55:26,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:54162 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:55:26,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:26,560 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972fbe430e18c5d163e7/lava_callback.json.gz 2026-03-28 08:55:26,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:26,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972fbe430e18c5d163e7 HTTP/1.1" 200 None 2026-03-28 08:55:26,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:26,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:26,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:26,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7972fbe430e18c5d163e7 HTTP/1.1" 200 None 2026-03-28 08:55:26,694 - lava_callback - INFO - Completed processing callback for node 69c7972fbe430e18c5d163e7 2026-03-28 08:55:27,812 - lava_callback - INFO - Received callback for node 69c79736be430e18c5d163fe INFO: 172.20.0.1:54178 - "POST /node/69c79736be430e18c5d163fe HTTP/1.0" 202 Accepted 2026-03-28 08:55:27,818 - lava_callback - INFO - Starting async job submit for node 69c79736be430e18c5d163fe 2026-03-28 08:55:27,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:27,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79736be430e18c5d163fe HTTP/1.1" 200 None 2026-03-28 08:55:27,856 - lava_callback - DEBUG - Processing node 69c79736be430e18c5d163fe: baseline-nfs-arm64 2026-03-28 08:55:28,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:28,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:28,376 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79736be430e18c5d163fe/log.txt.gz 2026-03-28 08:55:28,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:28,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:28,772 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79736be430e18c5d163fe/lava-logs.yaml 2026-03-28 08:55:28,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:28,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:28,971 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79736be430e18c5d163fe/lava_callback.json.gz 2026-03-28 08:55:28,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:29,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79736be430e18c5d163fe HTTP/1.1" 200 None 2026-03-28 08:55:29,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:29,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:29,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:29,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79736be430e18c5d163fe HTTP/1.1" 200 None 2026-03-28 08:55:29,055 - lava_callback - INFO - Completed processing callback for node 69c79736be430e18c5d163fe 2026-03-28 08:55:29,643 - lava_callback - INFO - Received callback for node 69c79727be430e18c5d163b5 INFO: 172.20.0.1:54182 - "POST /node/69c79727be430e18c5d163b5 HTTP/1.0" 202 Accepted 2026-03-28 08:55:29,649 - lava_callback - INFO - Starting async job submit for node 69c79727be430e18c5d163b5 2026-03-28 08:55:29,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:29,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79727be430e18c5d163b5 HTTP/1.1" 200 None 2026-03-28 08:55:29,691 - lava_callback - DEBUG - Processing node 69c79727be430e18c5d163b5: baseline-arm64 2026-03-28 08:55:29,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:30,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:30,103 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79727be430e18c5d163b5/log.txt.gz 2026-03-28 08:55:30,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:30,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:30,514 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79727be430e18c5d163b5/lava-logs.yaml 2026-03-28 08:55:30,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:30,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:30,660 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79727be430e18c5d163b5/lava_callback.json.gz 2026-03-28 08:55:30,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:30,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79727be430e18c5d163b5 HTTP/1.1" 200 None 2026-03-28 08:55:30,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:30,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:30,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:30,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79727be430e18c5d163b5 HTTP/1.1" 200 None 2026-03-28 08:55:30,762 - lava_callback - INFO - Completed processing callback for node 69c79727be430e18c5d163b5 2026-03-28 08:55:32,789 - lava_callback - INFO - Received callback for node 69c79731be430e18c5d163ea INFO: 172.20.0.1:58988 - "POST /node/69c79731be430e18c5d163ea HTTP/1.0" 202 Accepted 2026-03-28 08:55:32,796 - lava_callback - INFO - Starting async job submit for node 69c79731be430e18c5d163ea 2026-03-28 08:55:32,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:32,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79731be430e18c5d163ea HTTP/1.1" 200 None 2026-03-28 08:55:32,833 - lava_callback - DEBUG - Processing node 69c79731be430e18c5d163ea: baseline-nfs-arm64 2026-03-28 08:55:33,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:33,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:33,432 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79731be430e18c5d163ea/log.txt.gz 2026-03-28 08:55:33,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:33,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:33,910 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79731be430e18c5d163ea/lava-logs.yaml 2026-03-28 08:55:33,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:34,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:34,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79731be430e18c5d163ea/lava_callback.json.gz 2026-03-28 08:55:34,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:34,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79731be430e18c5d163ea HTTP/1.1" 200 None 2026-03-28 08:55:34,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:34,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:34,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:34,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79731be430e18c5d163ea HTTP/1.1" 200 None 2026-03-28 08:55:34,168 - lava_callback - INFO - Completed processing callback for node 69c79731be430e18c5d163ea INFO: 172.20.0.1:59000 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:55:36,597 - lava_callback - INFO - Received callback for node 69c79725be430e18c5d163ae INFO: 172.20.0.1:59004 - "POST /node/69c79725be430e18c5d163ae HTTP/1.0" 202 Accepted 2026-03-28 08:55:36,603 - lava_callback - INFO - Starting async job submit for node 69c79725be430e18c5d163ae 2026-03-28 08:55:36,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:36,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79725be430e18c5d163ae HTTP/1.1" 200 None 2026-03-28 08:55:36,639 - lava_callback - DEBUG - Processing node 69c79725be430e18c5d163ae: baseline-arm64 2026-03-28 08:55:36,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:37,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:37,048 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79725be430e18c5d163ae/log.txt.gz 2026-03-28 08:55:37,222 - lava_callback - INFO - Received callback for node 69c79733be430e18c5d163f1 INFO: 172.20.0.1:59006 - "POST /node/69c79733be430e18c5d163f1 HTTP/1.0" 202 Accepted 2026-03-28 08:55:37,233 - lava_callback - INFO - Starting async job submit for node 69c79733be430e18c5d163f1 2026-03-28 08:55:37,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:37,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:37,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79733be430e18c5d163f1 HTTP/1.1" 200 None 2026-03-28 08:55:37,295 - lava_callback - DEBUG - Processing node 69c79733be430e18c5d163f1: baseline-nfs-arm64 2026-03-28 08:55:37,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:37,573 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79725be430e18c5d163ae/lava-logs.yaml 2026-03-28 08:55:37,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:37,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:37,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:37,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79725be430e18c5d163ae/lava_callback.json.gz 2026-03-28 08:55:37,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:37,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:37,838 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79733be430e18c5d163f1/log.txt.gz 2026-03-28 08:55:37,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:37,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79725be430e18c5d163ae HTTP/1.1" 200 None 2026-03-28 08:55:37,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:37,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-28 08:55:38,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:38,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:38,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:38,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79725be430e18c5d163ae HTTP/1.1" 200 None 2026-03-28 08:55:38,072 - lava_callback - INFO - Completed processing callback for node 69c79725be430e18c5d163ae 2026-03-28 08:55:38,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:38,319 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79733be430e18c5d163f1/lava-logs.yaml 2026-03-28 08:55:38,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:38,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:38,484 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79733be430e18c5d163f1/lava_callback.json.gz 2026-03-28 08:55:38,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:38,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79733be430e18c5d163f1 HTTP/1.1" 200 None 2026-03-28 08:55:38,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:38,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:38,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:38,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79733be430e18c5d163f1 HTTP/1.1" 200 None 2026-03-28 08:55:38,571 - lava_callback - INFO - Completed processing callback for node 69c79733be430e18c5d163f1 INFO: 172.20.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58274 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:55:46,937 - lava_callback - INFO - Received callback for node 69c7975abe430e18c5d16486 INFO: 172.20.0.1:58276 - "POST /node/69c7975abe430e18c5d16486 HTTP/1.0" 202 Accepted 2026-03-28 08:55:46,943 - lava_callback - INFO - Starting async job submit for node 69c7975abe430e18c5d16486 2026-03-28 08:55:46,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:46,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975abe430e18c5d16486 HTTP/1.1" 200 None 2026-03-28 08:55:46,984 - lava_callback - DEBUG - Processing node 69c7975abe430e18c5d16486: baseline-arm64-pengutronix 2026-03-28 08:55:47,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:47,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:47,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975abe430e18c5d16486/log.txt.gz 2026-03-28 08:55:47,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:47,483 - lava_callback - INFO - Received callback for node 69c79743be430e18c5d16424 INFO: 172.20.0.1:58292 - "POST /node/69c79743be430e18c5d16424 HTTP/1.0" 202 Accepted 2026-03-28 08:55:47,490 - lava_callback - INFO - Starting async job submit for node 69c79743be430e18c5d16424 2026-03-28 08:55:47,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:47,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79743be430e18c5d16424 HTTP/1.1" 200 None 2026-03-28 08:55:47,531 - lava_callback - DEBUG - Processing node 69c79743be430e18c5d16424: baseline-arm64-broonie 2026-03-28 08:55:47,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:47,662 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975abe430e18c5d16486/lava-logs.yaml 2026-03-28 08:55:47,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:47,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:47,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:47,896 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79743be430e18c5d16424/log.txt.gz 2026-03-28 08:55:47,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:47,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975abe430e18c5d16486/lava_callback.json.gz 2026-03-28 08:55:48,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:48,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:48,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975abe430e18c5d16486 HTTP/1.1" 200 None 2026-03-28 08:55:48,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:48,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:48,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:48,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7975abe430e18c5d16486 HTTP/1.1" 200 None 2026-03-28 08:55:48,137 - lava_callback - INFO - Completed processing callback for node 69c7975abe430e18c5d16486 2026-03-28 08:55:48,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:48,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79743be430e18c5d16424/lava-logs.yaml 2026-03-28 08:55:48,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:48,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:48,499 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79743be430e18c5d16424/lava_callback.json.gz 2026-03-28 08:55:48,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:48,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79743be430e18c5d16424 HTTP/1.1" 200 None 2026-03-28 08:55:48,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:48,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:48,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:48,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79743be430e18c5d16424 HTTP/1.1" 200 None 2026-03-28 08:55:48,595 - lava_callback - INFO - Completed processing callback for node 69c79743be430e18c5d16424 2026-03-28 08:55:48,995 - lava_callback - INFO - Received callback for node 69c7974cbe430e18c5d16455 INFO: 172.20.0.1:58308 - "POST /node/69c7974cbe430e18c5d16455 HTTP/1.0" 202 Accepted 2026-03-28 08:55:49,002 - lava_callback - INFO - Starting async job submit for node 69c7974cbe430e18c5d16455 2026-03-28 08:55:49,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:49,032 - lava_callback - INFO - Received callback for node 69c7975dbe430e18c5d16495 INFO: 172.20.0.1:58318 - "POST /node/69c7975dbe430e18c5d16495 HTTP/1.0" 202 Accepted 2026-03-28 08:55:49,042 - lava_callback - INFO - Starting async job submit for node 69c7975dbe430e18c5d16495 2026-03-28 08:55:49,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974cbe430e18c5d16455 HTTP/1.1" 200 None 2026-03-28 08:55:49,078 - lava_callback - DEBUG - Processing node 69c7974cbe430e18c5d16455: baseline-arm64-broonie 2026-03-28 08:55:49,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:49,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975dbe430e18c5d16495 HTTP/1.1" 200 None 2026-03-28 08:55:49,200 - lava_callback - DEBUG - Processing node 69c7975dbe430e18c5d16495: baseline-arm64-foundriesio 2026-03-28 08:55:49,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:49,612 - lava_callback - INFO - Received callback for node 69c7975ebe430e18c5d16499 INFO: 172.20.0.1:58324 - "POST /node/69c7975ebe430e18c5d16499 HTTP/1.0" 202 Accepted 2026-03-28 08:55:49,623 - lava_callback - INFO - Starting async job submit for node 69c7975ebe430e18c5d16499 2026-03-28 08:55:49,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:49,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:49,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975ebe430e18c5d16499 HTTP/1.1" 200 None 2026-03-28 08:55:49,688 - lava_callback - DEBUG - Processing node 69c7975ebe430e18c5d16499: baseline-arm64-foundriesio 2026-03-28 08:55:49,862 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:49,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974cbe430e18c5d16455/log.txt.gz 2026-03-28 08:55:49,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:49,957 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c7975dbe430e18c5d16495/log.txt.gz 2026-03-28 08:55:50,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:50,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:50,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:50,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:50,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c7975dbe430e18c5d16495/lava-logs.yaml 2026-03-28 08:55:50,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:50,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:50,836 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974cbe430e18c5d16455/lava-logs.yaml 2026-03-28 08:55:50,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:50,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:50,898 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c7975ebe430e18c5d16499/log.txt.gz 2026-03-28 08:55:50,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:51,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c7975dbe430e18c5d16495/lava_callback.json.gz 2026-03-28 08:55:51,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:51,104 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974cbe430e18c5d16455/lava_callback.json.gz 2026-03-28 08:55:51,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975dbe430e18c5d16495 HTTP/1.1" 200 None 2026-03-28 08:55:51,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974cbe430e18c5d16455 HTTP/1.1" 200 None 2026-03-28 08:55:51,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:51,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:51,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:51,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7975dbe430e18c5d16495 HTTP/1.1" 200 None 2026-03-28 08:55:51,304 - lava_callback - INFO - Completed processing callback for node 69c7975dbe430e18c5d16495 2026-03-28 08:55:51,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7974cbe430e18c5d16455 HTTP/1.1" 200 None 2026-03-28 08:55:51,388 - lava_callback - INFO - Completed processing callback for node 69c7974cbe430e18c5d16455 2026-03-28 08:55:51,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:51,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c7975ebe430e18c5d16499/lava-logs.yaml 2026-03-28 08:55:51,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:51,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:51,807 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c7975ebe430e18c5d16499/lava_callback.json.gz 2026-03-28 08:55:51,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975ebe430e18c5d16499 HTTP/1.1" 200 None 2026-03-28 08:55:51,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:51,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:51,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7975ebe430e18c5d16499 HTTP/1.1" 200 None 2026-03-28 08:55:51,919 - lava_callback - INFO - Completed processing callback for node 69c7975ebe430e18c5d16499 2026-03-28 08:55:52,083 - lava_callback - INFO - Received callback for node 69c7972dbe430e18c5d163c3 INFO: 172.20.0.1:58086 - "POST /node/69c7972dbe430e18c5d163c3 HTTP/1.0" 202 Accepted 2026-03-28 08:55:52,094 - lava_callback - INFO - Starting async job submit for node 69c7972dbe430e18c5d163c3 2026-03-28 08:55:52,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:52,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972dbe430e18c5d163c3 HTTP/1.1" 200 None 2026-03-28 08:55:52,161 - lava_callback - DEBUG - Processing node 69c7972dbe430e18c5d163c3: baseline-nfs-arm64 2026-03-28 08:55:52,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:53,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:53,276 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972dbe430e18c5d163c3/log.txt.gz 2026-03-28 08:55:53,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:53,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:53,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972dbe430e18c5d163c3/lava-logs.yaml 2026-03-28 08:55:53,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:53,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:53,860 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972dbe430e18c5d163c3/lava_callback.json.gz 2026-03-28 08:55:53,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:53,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972dbe430e18c5d163c3 HTTP/1.1" 200 None 2026-03-28 08:55:53,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:53,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:53,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:53,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7972dbe430e18c5d163c3 HTTP/1.1" 200 None 2026-03-28 08:55:53,943 - lava_callback - INFO - Completed processing callback for node 69c7972dbe430e18c5d163c3 2026-03-28 08:55:56,481 - lava_callback - INFO - Received callback for node 69c7971fbe430e18c5d1639f INFO: 172.20.0.1:58098 - "POST /node/69c7971fbe430e18c5d1639f HTTP/1.0" 202 Accepted 2026-03-28 08:55:56,488 - lava_callback - INFO - Starting async job submit for node 69c7971fbe430e18c5d1639f 2026-03-28 08:55:56,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:55:56,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971fbe430e18c5d1639f HTTP/1.1" 200 None 2026-03-28 08:55:56,537 - lava_callback - DEBUG - Processing node 69c7971fbe430e18c5d1639f: baseline-arm64 2026-03-28 08:55:56,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:56,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:56,988 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971fbe430e18c5d1639f/log.txt.gz 2026-03-28 08:55:57,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:57,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:57,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971fbe430e18c5d1639f/lava-logs.yaml 2026-03-28 08:55:57,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:57,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:57,526 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971fbe430e18c5d1639f/lava_callback.json.gz 2026-03-28 08:55:57,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:57,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971fbe430e18c5d1639f HTTP/1.1" 200 None 2026-03-28 08:55:57,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:57,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:57,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:57,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7971fbe430e18c5d1639f HTTP/1.1" 200 None 2026-03-28 08:55:57,716 - lava_callback - INFO - Completed processing callback for node 69c7971fbe430e18c5d1639f 2026-03-28 08:55:57,734 - lava_callback - INFO - Received callback for node 69c79738be430e18c5d16405 INFO: 172.20.0.1:58112 - "POST /node/69c79738be430e18c5d16405 HTTP/1.0" 202 Accepted 2026-03-28 08:55:57,746 - lava_callback - INFO - Starting async job submit for node 69c79738be430e18c5d16405 2026-03-28 08:55:57,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:57,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79738be430e18c5d16405 HTTP/1.1" 200 None 2026-03-28 08:55:57,831 - lava_callback - DEBUG - Processing node 69c79738be430e18c5d16405: baseline-nfs-arm64 2026-03-28 08:55:58,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:58,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:58,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79738be430e18c5d16405/log.txt.gz 2026-03-28 08:55:58,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:58,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:58,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79738be430e18c5d16405/lava-logs.yaml 2026-03-28 08:55:58,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:55:58,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:55:58,929 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79738be430e18c5d16405/lava_callback.json.gz 2026-03-28 08:55:58,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:58,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79738be430e18c5d16405 HTTP/1.1" 200 None 2026-03-28 08:55:58,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:58,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:55:58,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:55:59,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79738be430e18c5d16405 HTTP/1.1" 200 None 2026-03-28 08:55:59,014 - lava_callback - INFO - Completed processing callback for node 69c79738be430e18c5d16405 2026-03-28 08:56:01,516 - lava_callback - INFO - Received callback for node 69c7975bbe430e18c5d1648b INFO: 172.20.0.1:39508 - "POST /node/69c7975bbe430e18c5d1648b HTTP/1.0" 202 Accepted 2026-03-28 08:56:01,522 - lava_callback - INFO - Starting async job submit for node 69c7975bbe430e18c5d1648b 2026-03-28 08:56:01,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:01,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975bbe430e18c5d1648b HTTP/1.1" 200 None 2026-03-28 08:56:01,556 - lava_callback - DEBUG - Processing node 69c7975bbe430e18c5d1648b: baseline-arm64-pengutronix 2026-03-28 08:56:01,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:01,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:01,862 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975bbe430e18c5d1648b/log.txt.gz 2026-03-28 08:56:01,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:02,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:02,156 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975bbe430e18c5d1648b/lava-logs.yaml 2026-03-28 08:56:02,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:02,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:02,296 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975bbe430e18c5d1648b/lava_callback.json.gz 2026-03-28 08:56:02,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:02,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975bbe430e18c5d1648b HTTP/1.1" 200 None 2026-03-28 08:56:02,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:02,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:02,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:02,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7975bbe430e18c5d1648b HTTP/1.1" 200 None 2026-03-28 08:56:02,447 - lava_callback - INFO - Completed processing callback for node 69c7975bbe430e18c5d1648b INFO: 172.20.0.1:39518 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:56:07,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:07,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-28 08:56:08,519 - lava_callback - INFO - Received callback for node 69c793a3be430e18c5d1600a INFO: 172.20.0.1:39528 - "POST /node/69c793a3be430e18c5d1600a HTTP/1.0" 202 Accepted 2026-03-28 08:56:08,526 - lava_callback - INFO - Starting async job submit for node 69c793a3be430e18c5d1600a 2026-03-28 08:56:08,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:08,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a3be430e18c5d1600a HTTP/1.1" 200 None 2026-03-28 08:56:08,609 - lava_callback - DEBUG - Processing node 69c793a3be430e18c5d1600a: baseline-arm 2026-03-28 08:56:09,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:09,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:09,343 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a3be430e18c5d1600a/log.txt.gz 2026-03-28 08:56:09,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:09,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:09,915 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a3be430e18c5d1600a/lava-logs.yaml 2026-03-28 08:56:09,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:10,103 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:10,105 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c793a3be430e18c5d1600a/lava_callback.json.gz 2026-03-28 08:56:10,147 - lava_callback - INFO - Received callback for node 69c79718be430e18c5d1638d INFO: 172.20.0.1:56588 - "POST /node/69c79718be430e18c5d1638d HTTP/1.0" 202 Accepted 2026-03-28 08:56:10,159 - lava_callback - INFO - Starting async job submit for node 69c79718be430e18c5d1638d 2026-03-28 08:56:10,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 DEBUG: Setup failed for 69c793a3be430e18c5d1600a. Transitting job node result: incomplete -> fail 2026-03-28 08:56:10,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:10,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79718be430e18c5d1638d HTTP/1.1" 200 None 2026-03-28 08:56:10,282 - lava_callback - DEBUG - Processing node 69c79718be430e18c5d1638d: baseline-arm64 2026-03-28 08:56:10,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793a3be430e18c5d1600a HTTP/1.1" 200 None 2026-03-28 08:56:10,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:10,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:56:10,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:10,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:10,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793a3be430e18c5d1600a HTTP/1.1" 200 None 2026-03-28 08:56:10,692 - lava_callback - INFO - Completed processing callback for node 69c793a3be430e18c5d1600a 2026-03-28 08:56:10,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:10,762 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79718be430e18c5d1638d/log.txt.gz 2026-03-28 08:56:10,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:11,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:11,099 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79718be430e18c5d1638d/lava-logs.yaml 2026-03-28 08:56:11,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:11,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:11,247 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79718be430e18c5d1638d/lava_callback.json.gz 2026-03-28 08:56:11,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:11,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79718be430e18c5d1638d HTTP/1.1" 200 None 2026-03-28 08:56:11,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:11,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:11,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:11,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79718be430e18c5d1638d HTTP/1.1" 200 None 2026-03-28 08:56:11,402 - lava_callback - INFO - Completed processing callback for node 69c79718be430e18c5d1638d 2026-03-28 08:56:13,994 - lava_callback - INFO - Received callback for node 69c7972abe430e18c5d163bb INFO: 172.20.0.1:56600 - "POST /node/69c7972abe430e18c5d163bb HTTP/1.0" 202 Accepted 2026-03-28 08:56:14,001 - lava_callback - INFO - Starting async job submit for node 69c7972abe430e18c5d163bb 2026-03-28 08:56:14,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:14,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972abe430e18c5d163bb HTTP/1.1" 200 None 2026-03-28 08:56:14,044 - lava_callback - DEBUG - Processing node 69c7972abe430e18c5d163bb: baseline-nfs-arm64 2026-03-28 08:56:14,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:14,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:14,491 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972abe430e18c5d163bb/log.txt.gz 2026-03-28 08:56:14,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:14,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:14,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972abe430e18c5d163bb/lava-logs.yaml 2026-03-28 08:56:14,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:15,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:15,025 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972abe430e18c5d163bb/lava_callback.json.gz 2026-03-28 08:56:15,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:15,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972abe430e18c5d163bb HTTP/1.1" 200 None 2026-03-28 08:56:15,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:15,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:15,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:15,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7972abe430e18c5d163bb HTTP/1.1" 200 None 2026-03-28 08:56:15,111 - lava_callback - INFO - Completed processing callback for node 69c7972abe430e18c5d163bb INFO: 172.20.0.1:56604 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 08:56:15,972 - lava_callback - INFO - Received callback for node 69c7974fbe430e18c5d16465 INFO: 172.20.0.1:56608 - "POST /node/69c7974fbe430e18c5d16465 HTTP/1.0" 202 Accepted 2026-03-28 08:56:15,978 - lava_callback - INFO - Starting async job submit for node 69c7974fbe430e18c5d16465 2026-03-28 08:56:16,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:16,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974fbe430e18c5d16465 HTTP/1.1" 200 None 2026-03-28 08:56:16,015 - lava_callback - DEBUG - Processing node 69c7974fbe430e18c5d16465: baseline-arm64-broonie 2026-03-28 08:56:16,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:16,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:16,379 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974fbe430e18c5d16465/log.txt.gz INFO: 172.20.0.1:56622 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:56:16,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:16,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:16,710 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974fbe430e18c5d16465/lava-logs.yaml 2026-03-28 08:56:16,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:16,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:16,852 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974fbe430e18c5d16465/lava_callback.json.gz 2026-03-28 08:56:16,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:16,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974fbe430e18c5d16465 HTTP/1.1" 200 None 2026-03-28 08:56:16,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:16,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:16,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:16,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7974fbe430e18c5d16465 HTTP/1.1" 200 None 2026-03-28 08:56:16,930 - lava_callback - INFO - Completed processing callback for node 69c7974fbe430e18c5d16465 INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:56:27,413 - lava_callback - INFO - Received callback for node 69c79751be430e18c5d16468 INFO: 172.20.0.1:60136 - "POST /node/69c79751be430e18c5d16468 HTTP/1.0" 202 Accepted 2026-03-28 08:56:27,420 - lava_callback - INFO - Starting async job submit for node 69c79751be430e18c5d16468 2026-03-28 08:56:27,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:27,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79751be430e18c5d16468 HTTP/1.1" 200 None 2026-03-28 08:56:27,457 - lava_callback - DEBUG - Processing node 69c79751be430e18c5d16468: baseline-arm64-broonie 2026-03-28 08:56:27,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:27,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:27,825 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79751be430e18c5d16468/log.txt.gz 2026-03-28 08:56:27,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:28,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:28,173 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79751be430e18c5d16468/lava-logs.yaml 2026-03-28 08:56:28,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:28,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:28,309 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79751be430e18c5d16468/lava_callback.json.gz 2026-03-28 08:56:28,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:28,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79751be430e18c5d16468 HTTP/1.1" 200 None 2026-03-28 08:56:28,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:28,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:28,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:28,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79751be430e18c5d16468 HTTP/1.1" 200 None 2026-03-28 08:56:28,400 - lava_callback - INFO - Completed processing callback for node 69c79751be430e18c5d16468 2026-03-28 08:56:35,879 - lava_callback - INFO - Received callback for node 69c79749be430e18c5d1642e INFO: 172.20.0.1:52772 - "POST /node/69c79749be430e18c5d1642e HTTP/1.0" 202 Accepted 2026-03-28 08:56:35,885 - lava_callback - INFO - Starting async job submit for node 69c79749be430e18c5d1642e 2026-03-28 08:56:35,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:35,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79749be430e18c5d1642e HTTP/1.1" 200 None 2026-03-28 08:56:35,923 - lava_callback - DEBUG - Processing node 69c79749be430e18c5d1642e: baseline-arm64-broonie 2026-03-28 08:56:36,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:36,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:36,329 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79749be430e18c5d1642e/log.txt.gz 2026-03-28 08:56:36,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:56:36,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:36,674 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79749be430e18c5d1642e/lava-logs.yaml 2026-03-28 08:56:36,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:36,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:36,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79749be430e18c5d1642e/lava_callback.json.gz 2026-03-28 08:56:36,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:36,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79749be430e18c5d1642e HTTP/1.1" 200 None 2026-03-28 08:56:36,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:36,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:36,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:36,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79749be430e18c5d1642e HTTP/1.1" 200 None 2026-03-28 08:56:36,924 - lava_callback - INFO - Completed processing callback for node 69c79749be430e18c5d1642e 2026-03-28 08:56:37,369 - lava_callback - INFO - Received callback for node 69c7974dbe430e18c5d16462 INFO: 172.20.0.1:52782 - "POST /node/69c7974dbe430e18c5d16462 HTTP/1.0" 202 Accepted 2026-03-28 08:56:37,375 - lava_callback - INFO - Starting async job submit for node 69c7974dbe430e18c5d16462 2026-03-28 08:56:37,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:37,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974dbe430e18c5d16462 HTTP/1.1" 200 None 2026-03-28 08:56:37,413 - lava_callback - DEBUG - Processing node 69c7974dbe430e18c5d16462: baseline-arm64-broonie 2026-03-28 08:56:37,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:37,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:37,820 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974dbe430e18c5d16462/log.txt.gz 2026-03-28 08:56:37,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:37,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:37,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-28 08:56:38,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:38,170 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974dbe430e18c5d16462/lava-logs.yaml 2026-03-28 08:56:38,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:38,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:38,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974dbe430e18c5d16462/lava_callback.json.gz 2026-03-28 08:56:38,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:38,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974dbe430e18c5d16462 HTTP/1.1" 200 None 2026-03-28 08:56:38,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:38,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:38,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:38,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7974dbe430e18c5d16462 HTTP/1.1" 200 None 2026-03-28 08:56:38,424 - lava_callback - INFO - Completed processing callback for node 69c7974dbe430e18c5d16462 INFO: 172.20.0.1:58620 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:56:42,893 - lava_callback - INFO - Received callback for node 69c79756be430e18c5d1647d INFO: 172.20.0.1:58636 - "POST /node/69c79756be430e18c5d1647d HTTP/1.0" 202 Accepted 2026-03-28 08:56:42,902 - lava_callback - INFO - Starting async job submit for node 69c79756be430e18c5d1647d 2026-03-28 08:56:42,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:42,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79756be430e18c5d1647d HTTP/1.1" 200 None 2026-03-28 08:56:42,957 - lava_callback - DEBUG - Processing node 69c79756be430e18c5d1647d: baseline-arm64-clabbe 2026-03-28 08:56:43,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:43,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:43,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c79756be430e18c5d1647d/log.txt.gz 2026-03-28 08:56:43,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:43,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:43,651 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c79756be430e18c5d1647d/lava-logs.yaml 2026-03-28 08:56:43,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:43,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:43,798 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c79756be430e18c5d1647d/lava_callback.json.gz 2026-03-28 08:56:43,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:43,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79756be430e18c5d1647d HTTP/1.1" 200 None 2026-03-28 08:56:43,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:43,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:43,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:43,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79756be430e18c5d1647d HTTP/1.1" 200 None 2026-03-28 08:56:43,880 - lava_callback - INFO - Completed processing callback for node 69c79756be430e18c5d1647d INFO: 172.20.0.1:58648 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:56:54,044 - lava_callback - INFO - Received callback for node 69c7975cbe430e18c5d16490 INFO: 172.20.0.1:46840 - "POST /node/69c7975cbe430e18c5d16490 HTTP/1.0" 202 Accepted 2026-03-28 08:56:54,093 - lava_callback - INFO - Starting async job submit for node 69c7975cbe430e18c5d16490 2026-03-28 08:56:54,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:54,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975cbe430e18c5d16490 HTTP/1.1" 200 None 2026-03-28 08:56:54,129 - lava_callback - DEBUG - Processing node 69c7975cbe430e18c5d16490: baseline-arm64-pengutronix 2026-03-28 08:56:54,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:54,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:54,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975cbe430e18c5d16490/log.txt.gz 2026-03-28 08:56:54,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:54,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:54,786 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975cbe430e18c5d16490/lava-logs.yaml 2026-03-28 08:56:54,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:56:54,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:56:54,931 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c7975cbe430e18c5d16490/lava_callback.json.gz 2026-03-28 08:56:54,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:54,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7975cbe430e18c5d16490 HTTP/1.1" 200 None 2026-03-28 08:56:54,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:54,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:56:54,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:56:55,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7975cbe430e18c5d16490 HTTP/1.1" 200 None 2026-03-28 08:56:55,009 - lava_callback - INFO - Completed processing callback for node 69c7975cbe430e18c5d16490 INFO: 172.20.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50512 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:57:07,688 - lava_callback - INFO - Received callback for node 69c79745be430e18c5d16427 INFO: 172.20.0.1:50518 - "POST /node/69c79745be430e18c5d16427 HTTP/1.0" 202 Accepted 2026-03-28 08:57:07,694 - lava_callback - INFO - Starting async job submit for node 69c79745be430e18c5d16427 2026-03-28 08:57:07,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:07,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79745be430e18c5d16427 HTTP/1.1" 200 None 2026-03-28 08:57:07,740 - lava_callback - DEBUG - Processing node 69c79745be430e18c5d16427: baseline-arm64-broonie 2026-03-28 08:57:07,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:08,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:08,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 08:57:08,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:08,112 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79745be430e18c5d16427/log.txt.gz 2026-03-28 08:57:08,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:08,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:08,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79745be430e18c5d16427/lava-logs.yaml 2026-03-28 08:57:08,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:08,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:08,581 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79745be430e18c5d16427/lava_callback.json.gz 2026-03-28 08:57:08,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:08,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79745be430e18c5d16427 HTTP/1.1" 200 None 2026-03-28 08:57:08,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:08,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:57:08,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:08,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79745be430e18c5d16427 HTTP/1.1" 200 None 2026-03-28 08:57:08,670 - lava_callback - INFO - Completed processing callback for node 69c79745be430e18c5d16427 2026-03-28 08:57:12,799 - lava_callback - INFO - Received callback for node 69c4f4e7f63ce05331e2aee1 INFO: 172.20.0.1:37640 - "POST /node/69c4f4e7f63ce05331e2aee1 HTTP/1.0" 202 Accepted 2026-03-28 08:57:12,805 - lava_callback - INFO - Starting async job submit for node 69c4f4e7f63ce05331e2aee1 2026-03-28 08:57:12,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:12,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e7f63ce05331e2aee1 HTTP/1.1" 200 None 2026-03-28 08:57:12,822 - lava_callback - DEBUG - Processing node 69c4f4e7f63ce05331e2aee1: baseline-arm64-baylibre 2026-03-28 08:57:12,822 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f4e7f63ce05331e2aee1 2026-03-28 08:57:12,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:12,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e7f63ce05331e2aee1 HTTP/1.1" 200 None 2026-03-28 08:57:12,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:12,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:12,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:12,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4e7f63ce05331e2aee1 HTTP/1.1" 200 None 2026-03-28 08:57:12,887 - lava_callback - ERROR - Error processing callback for node 69c4f4e7f63ce05331e2aee1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:13,468 - lava_callback - INFO - Received callback for node 69c4f4d5f63ce05331e2aebb INFO: 172.20.0.1:37652 - "POST /node/69c4f4d5f63ce05331e2aebb HTTP/1.0" 202 Accepted 2026-03-28 08:57:13,473 - lava_callback - INFO - Starting async job submit for node 69c4f4d5f63ce05331e2aebb 2026-03-28 08:57:13,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:13,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d5f63ce05331e2aebb HTTP/1.1" 200 None 2026-03-28 08:57:13,489 - lava_callback - DEBUG - Processing node 69c4f4d5f63ce05331e2aebb: baseline-arm-baylibre 2026-03-28 08:57:13,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f4d5f63ce05331e2aebb 2026-03-28 08:57:13,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:13,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4d5f63ce05331e2aebb HTTP/1.1" 200 None 2026-03-28 08:57:13,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:13,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-28 08:57:13,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:13,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4d5f63ce05331e2aebb HTTP/1.1" 200 None 2026-03-28 08:57:13,548 - lava_callback - ERROR - Error processing callback for node 69c4f4d5f63ce05331e2aebb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:14,139 - lava_callback - INFO - Received callback for node 69c4f4e5f63ce05331e2aede INFO: 172.20.0.1:37654 - "POST /node/69c4f4e5f63ce05331e2aede HTTP/1.0" 202 Accepted 2026-03-28 08:57:14,144 - lava_callback - INFO - Starting async job submit for node 69c4f4e5f63ce05331e2aede 2026-03-28 08:57:14,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:14,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e5f63ce05331e2aede HTTP/1.1" 200 None 2026-03-28 08:57:14,161 - lava_callback - DEBUG - Processing node 69c4f4e5f63ce05331e2aede: baseline-arm64-baylibre 2026-03-28 08:57:14,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f4e5f63ce05331e2aede 2026-03-28 08:57:14,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:14,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e5f63ce05331e2aede HTTP/1.1" 200 None 2026-03-28 08:57:14,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:14,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:14,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:14,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4e5f63ce05331e2aede HTTP/1.1" 200 None 2026-03-28 08:57:14,212 - lava_callback - ERROR - Error processing callback for node 69c4f4e5f63ce05331e2aede: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58570 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:57:32,738 - lava_callback - INFO - Received callback for node 69c4f4e8f63ce05331e2aee4 INFO: 172.20.0.1:40592 - "POST /node/69c4f4e8f63ce05331e2aee4 HTTP/1.0" 202 Accepted 2026-03-28 08:57:32,743 - lava_callback - INFO - Starting async job submit for node 69c4f4e8f63ce05331e2aee4 2026-03-28 08:57:32,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:32,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e8f63ce05331e2aee4 HTTP/1.1" 200 None 2026-03-28 08:57:32,759 - lava_callback - DEBUG - Processing node 69c4f4e8f63ce05331e2aee4: baseline-arm64-baylibre 2026-03-28 08:57:32,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f4e8f63ce05331e2aee4 2026-03-28 08:57:32,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:32,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4e8f63ce05331e2aee4 HTTP/1.1" 200 None 2026-03-28 08:57:32,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:32,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:32,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:32,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4e8f63ce05331e2aee4 HTTP/1.1" 200 None 2026-03-28 08:57:32,820 - lava_callback - ERROR - Error processing callback for node 69c4f4e8f63ce05331e2aee4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:33,391 - lava_callback - INFO - Received callback for node 69c4f4eef63ce05331e2aef5 INFO: 172.20.0.1:40608 - "POST /node/69c4f4eef63ce05331e2aef5 HTTP/1.0" 202 Accepted 2026-03-28 08:57:33,397 - lava_callback - INFO - Starting async job submit for node 69c4f4eef63ce05331e2aef5 2026-03-28 08:57:33,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:33,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4eef63ce05331e2aef5 HTTP/1.1" 200 None 2026-03-28 08:57:33,412 - lava_callback - DEBUG - Processing node 69c4f4eef63ce05331e2aef5: baseline-arm64-baylibre 2026-03-28 08:57:33,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f4eef63ce05331e2aef5 2026-03-28 08:57:33,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:33,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4eef63ce05331e2aef5 HTTP/1.1" 200 None 2026-03-28 08:57:33,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:33,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:33,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:33,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4eef63ce05331e2aef5 HTTP/1.1" 200 None 2026-03-28 08:57:33,476 - lava_callback - ERROR - Error processing callback for node 69c4f4eef63ce05331e2aef5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:34,055 - lava_callback - INFO - Received callback for node 69c4f4ebf63ce05331e2aeeb INFO: 172.20.0.1:40616 - "POST /node/69c4f4ebf63ce05331e2aeeb HTTP/1.0" 202 Accepted 2026-03-28 08:57:34,060 - lava_callback - INFO - Starting async job submit for node 69c4f4ebf63ce05331e2aeeb 2026-03-28 08:57:34,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4ebf63ce05331e2aeeb HTTP/1.1" 200 None 2026-03-28 08:57:34,076 - lava_callback - DEBUG - Processing node 69c4f4ebf63ce05331e2aeeb: baseline-arm64-baylibre 2026-03-28 08:57:34,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f4ebf63ce05331e2aeeb 2026-03-28 08:57:34,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4ebf63ce05331e2aeeb HTTP/1.1" 200 None 2026-03-28 08:57:34,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:34,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4ebf63ce05331e2aeeb HTTP/1.1" 200 None 2026-03-28 08:57:34,128 - lava_callback - ERROR - Error processing callback for node 69c4f4ebf63ce05331e2aeeb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:34,490 - lava_callback - INFO - Received callback for node 69c7971cbe430e18c5d16396 INFO: 172.20.0.1:40628 - "POST /node/69c7971cbe430e18c5d16396 HTTP/1.0" 202 Accepted 2026-03-28 08:57:34,510 - lava_callback - INFO - Starting async job submit for node 69c7971cbe430e18c5d16396 2026-03-28 08:57:34,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971cbe430e18c5d16396 HTTP/1.1" 200 None 2026-03-28 08:57:34,566 - lava_callback - DEBUG - Processing node 69c7971cbe430e18c5d16396: baseline-arm64 2026-03-28 08:57:34,759 - lava_callback - INFO - Received callback for node 69c4f4edf63ce05331e2aef2 INFO: 172.20.0.1:40632 - "POST /node/69c4f4edf63ce05331e2aef2 HTTP/1.0" 202 Accepted 2026-03-28 08:57:34,780 - lava_callback - INFO - Starting async job submit for node 69c4f4edf63ce05331e2aef2 2026-03-28 08:57:34,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4edf63ce05331e2aef2 HTTP/1.1" 200 None 2026-03-28 08:57:34,915 - lava_callback - DEBUG - Processing node 69c4f4edf63ce05331e2aef2: baseline-arm64-baylibre 2026-03-28 08:57:34,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f4edf63ce05331e2aef2 2026-03-28 08:57:34,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:34,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f4edf63ce05331e2aef2 HTTP/1.1" 200 None 2026-03-28 08:57:34,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:34,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:34,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f4edf63ce05331e2aef2 HTTP/1.1" 200 None 2026-03-28 08:57:34,983 - lava_callback - ERROR - Error processing callback for node 69c4f4edf63ce05331e2aef2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:35,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:35,139 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971cbe430e18c5d16396/log.txt.gz 2026-03-28 08:57:35,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:35,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:35,602 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971cbe430e18c5d16396/lava-logs.yaml 2026-03-28 08:57:35,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:35,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:35,773 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7971cbe430e18c5d16396/lava_callback.json.gz 2026-03-28 08:57:35,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:35,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7971cbe430e18c5d16396 HTTP/1.1" 200 None 2026-03-28 08:57:35,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:35,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:57:35,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:35,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7971cbe430e18c5d16396 HTTP/1.1" 200 None 2026-03-28 08:57:35,868 - lava_callback - INFO - Completed processing callback for node 69c7971cbe430e18c5d16396 INFO: 172.20.0.1:40642 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:57:38,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:38,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51964 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:57:43,754 - lava_callback - INFO - Received callback for node 69c4f502f63ce05331e2af3f INFO: 172.20.0.1:51966 - "POST /node/69c4f502f63ce05331e2af3f HTTP/1.0" 202 Accepted 2026-03-28 08:57:43,760 - lava_callback - INFO - Starting async job submit for node 69c4f502f63ce05331e2af3f 2026-03-28 08:57:43,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:43,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f502f63ce05331e2af3f HTTP/1.1" 200 None 2026-03-28 08:57:43,776 - lava_callback - DEBUG - Processing node 69c4f502f63ce05331e2af3f: baseline-arm64-clabbe 2026-03-28 08:57:43,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f502f63ce05331e2af3f 2026-03-28 08:57:43,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:43,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f502f63ce05331e2af3f HTTP/1.1" 200 None 2026-03-28 08:57:43,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:43,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:43,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:43,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f502f63ce05331e2af3f HTTP/1.1" 200 None 2026-03-28 08:57:43,831 - lava_callback - ERROR - Error processing callback for node 69c4f502f63ce05331e2af3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:44,433 - lava_callback - INFO - Received callback for node 69c4f504f63ce05331e2af42 INFO: 172.20.0.1:51970 - "POST /node/69c4f504f63ce05331e2af42 HTTP/1.0" 202 Accepted 2026-03-28 08:57:44,439 - lava_callback - INFO - Starting async job submit for node 69c4f504f63ce05331e2af42 2026-03-28 08:57:44,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:44,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f504f63ce05331e2af42 HTTP/1.1" 200 None 2026-03-28 08:57:44,458 - lava_callback - DEBUG - Processing node 69c4f504f63ce05331e2af42: baseline-arm64-clabbe 2026-03-28 08:57:44,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f504f63ce05331e2af42 2026-03-28 08:57:44,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:44,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f504f63ce05331e2af42 HTTP/1.1" 200 None 2026-03-28 08:57:44,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:44,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:44,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:44,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f504f63ce05331e2af42 HTTP/1.1" 200 None 2026-03-28 08:57:44,512 - lava_callback - ERROR - Error processing callback for node 69c4f504f63ce05331e2af42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:45,131 - lava_callback - INFO - Received callback for node 69c4f505f63ce05331e2af45 INFO: 172.20.0.1:51976 - "POST /node/69c4f505f63ce05331e2af45 HTTP/1.0" 202 Accepted 2026-03-28 08:57:45,136 - lava_callback - INFO - Starting async job submit for node 69c4f505f63ce05331e2af45 2026-03-28 08:57:45,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f505f63ce05331e2af45 HTTP/1.1" 200 None 2026-03-28 08:57:45,160 - lava_callback - DEBUG - Processing node 69c4f505f63ce05331e2af45: baseline-arm64-clabbe 2026-03-28 08:57:45,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f505f63ce05331e2af45 2026-03-28 08:57:45,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f505f63ce05331e2af45 HTTP/1.1" 200 None 2026-03-28 08:57:45,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:45,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f505f63ce05331e2af45 HTTP/1.1" 200 None 2026-03-28 08:57:45,220 - lava_callback - ERROR - Error processing callback for node 69c4f505f63ce05331e2af45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:57:45,832 - lava_callback - INFO - Received callback for node 69c4f506f63ce05331e2af48 INFO: 172.20.0.1:51986 - "POST /node/69c4f506f63ce05331e2af48 HTTP/1.0" 202 Accepted 2026-03-28 08:57:45,838 - lava_callback - INFO - Starting async job submit for node 69c4f506f63ce05331e2af48 2026-03-28 08:57:45,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f506f63ce05331e2af48 HTTP/1.1" 200 None 2026-03-28 08:57:45,858 - lava_callback - DEBUG - Processing node 69c4f506f63ce05331e2af48: baseline-arm64-clabbe 2026-03-28 08:57:45,858 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f506f63ce05331e2af48 2026-03-28 08:57:45,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f506f63ce05331e2af48 HTTP/1.1" 200 None 2026-03-28 08:57:45,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:57:45,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:45,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f506f63ce05331e2af48 HTTP/1.1" 200 None 2026-03-28 08:57:45,913 - lava_callback - ERROR - Error processing callback for node 69c4f506f63ce05331e2af48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39636 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:57:58,401 - lava_callback - INFO - Received callback for node 69c79805be430e18c5d165c2 INFO: 172.20.0.1:39642 - "POST /node/69c79805be430e18c5d165c2 HTTP/1.0" 202 Accepted 2026-03-28 08:57:58,406 - lava_callback - INFO - Starting async job submit for node 69c79805be430e18c5d165c2 2026-03-28 08:57:58,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:58,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79805be430e18c5d165c2 HTTP/1.1" 200 None 2026-03-28 08:57:58,425 - lava_callback - DEBUG - Processing node 69c79805be430e18c5d165c2: baseline-arm 2026-03-28 08:57:58,430 - lava_callback - WARNING - Failed to upload log 2026-03-28 08:57:58,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:58,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:58,578 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c79805be430e18c5d165c2/lava-logs.yaml 2026-03-28 08:57:58,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:57:58,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:57:58,708 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c79805be430e18c5d165c2/lava_callback.json.gz 2026-03-28 08:57:58,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:58,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79805be430e18c5d165c2 HTTP/1.1" 200 None 2026-03-28 08:57:58,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:58,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:57:58,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:57:58,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79805be430e18c5d165c2 HTTP/1.1" 200 None 2026-03-28 08:57:58,774 - lava_callback - INFO - Completed processing callback for node 69c79805be430e18c5d165c2 2026-03-28 08:58:03,721 - lava_callback - INFO - Received callback for node 69c7972cbe430e18c5d163c0 INFO: 172.20.0.1:33752 - "POST /node/69c7972cbe430e18c5d163c0 HTTP/1.0" 202 Accepted 2026-03-28 08:58:03,727 - lava_callback - INFO - Starting async job submit for node 69c7972cbe430e18c5d163c0 2026-03-28 08:58:03,743 - lava_callback - INFO - Received callback for node 69c4f508f63ce05331e2af4e INFO: 172.20.0.1:33768 - "POST /node/69c4f508f63ce05331e2af4e HTTP/1.0" 202 Accepted 2026-03-28 08:58:03,751 - lava_callback - INFO - Starting async job submit for node 69c4f508f63ce05331e2af4e 2026-03-28 08:58:03,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:03,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:03,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f508f63ce05331e2af4e HTTP/1.1" 200 None 2026-03-28 08:58:03,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972cbe430e18c5d163c0 HTTP/1.1" 200 None 2026-03-28 08:58:03,769 - lava_callback - DEBUG - Processing node 69c4f508f63ce05331e2af4e: baseline-arm64-clabbe 2026-03-28 08:58:03,769 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f508f63ce05331e2af4e 2026-03-28 08:58:03,770 - lava_callback - DEBUG - Processing node 69c7972cbe430e18c5d163c0: baseline-nfs-arm64 2026-03-28 08:58:03,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:03,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f508f63ce05331e2af4e HTTP/1.1" 200 None 2026-03-28 08:58:03,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:03,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:58:04,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:04,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f508f63ce05331e2af4e HTTP/1.1" 200 None 2026-03-28 08:58:04,131 - lava_callback - ERROR - Error processing callback for node 69c4f508f63ce05331e2af4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:58:04,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:04,426 - lava_callback - INFO - Received callback for node 69c4f509f63ce05331e2af56 INFO: 172.20.0.1:33772 - "POST /node/69c4f509f63ce05331e2af56 HTTP/1.0" 202 Accepted 2026-03-28 08:58:04,433 - lava_callback - INFO - Starting async job submit for node 69c4f509f63ce05331e2af56 2026-03-28 08:58:04,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:04,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f509f63ce05331e2af56 HTTP/1.1" 200 None 2026-03-28 08:58:04,450 - lava_callback - DEBUG - Processing node 69c4f509f63ce05331e2af56: baseline-arm64-clabbe 2026-03-28 08:58:04,450 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f509f63ce05331e2af56 2026-03-28 08:58:04,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:04,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:04,466 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972cbe430e18c5d163c0/log.txt.gz 2026-03-28 08:58:04,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f509f63ce05331e2af56 HTTP/1.1" 200 None 2026-03-28 08:58:04,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:04,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 08:58:04,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:04,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f509f63ce05331e2af56 HTTP/1.1" 200 None 2026-03-28 08:58:04,714 - lava_callback - ERROR - Error processing callback for node 69c4f509f63ce05331e2af56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 08:58:04,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:05,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:05,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972cbe430e18c5d163c0/lava-logs.yaml 2026-03-28 08:58:05,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:05,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:05,182 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972cbe430e18c5d163c0/lava_callback.json.gz 2026-03-28 08:58:05,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:05,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972cbe430e18c5d163c0 HTTP/1.1" 200 None 2026-03-28 08:58:05,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:05,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:58:05,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:05,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7972cbe430e18c5d163c0 HTTP/1.1" 200 None 2026-03-28 08:58:05,274 - lava_callback - INFO - Completed processing callback for node 69c7972cbe430e18c5d163c0 INFO: 172.20.0.1:33778 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:58:08,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:08,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47696 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:58:27,679 - lava_callback - INFO - Received callback for node 69c793e9be430e18c5d160f5 INFO: 172.20.0.1:47704 - "POST /node/69c793e9be430e18c5d160f5 HTTP/1.0" 202 Accepted 2026-03-28 08:58:27,686 - lava_callback - INFO - Starting async job submit for node 69c793e9be430e18c5d160f5 2026-03-28 08:58:27,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:27,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e9be430e18c5d160f5 HTTP/1.1" 200 None 2026-03-28 08:58:27,741 - lava_callback - DEBUG - Processing node 69c793e9be430e18c5d160f5: ltp-syscalls-ipc 2026-03-28 08:58:28,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:28,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:28,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c793e9be430e18c5d160f5/log.txt.gz 2026-03-28 08:58:28,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:28,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:28,851 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c793e9be430e18c5d160f5/lava-logs.yaml 2026-03-28 08:58:28,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:29,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:29,016 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c793e9be430e18c5d160f5/lava_callback.json.gz 2026-03-28 08:58:29,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:29,085 - lava_callback - INFO - Received callback for node 69c7974abe430e18c5d16435 2026-03-28 08:58:29,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793e9be430e18c5d160f5 HTTP/1.1" 200 None INFO: 172.20.0.1:47710 - "POST /node/69c7974abe430e18c5d16435 HTTP/1.0" 202 Accepted 2026-03-28 08:58:29,093 - lava_callback - INFO - Starting async job submit for node 69c7974abe430e18c5d16435 2026-03-28 08:58:29,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:29,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:29,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974abe430e18c5d16435 HTTP/1.1" 200 None 2026-03-28 08:58:29,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:58:29,134 - lava_callback - DEBUG - Processing node 69c7974abe430e18c5d16435: baseline-arm64-broonie 2026-03-28 08:58:29,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:29,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793e9be430e18c5d160f5 HTTP/1.1" 200 None 2026-03-28 08:58:29,297 - lava_callback - INFO - Completed processing callback for node 69c793e9be430e18c5d160f5 2026-03-28 08:58:29,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:29,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:29,667 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974abe430e18c5d16435/log.txt.gz 2026-03-28 08:58:29,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:30,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:30,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974abe430e18c5d16435/lava-logs.yaml 2026-03-28 08:58:30,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:30,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:30,196 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7974abe430e18c5d16435/lava_callback.json.gz 2026-03-28 08:58:30,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:30,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7974abe430e18c5d16435 HTTP/1.1" 200 None 2026-03-28 08:58:30,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:30,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:58:30,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:30,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7974abe430e18c5d16435 HTTP/1.1" 200 None 2026-03-28 08:58:30,278 - lava_callback - INFO - Completed processing callback for node 69c7974abe430e18c5d16435 INFO: 172.20.0.1:49548 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:58:38,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:38,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34816 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:58:55,577 - lava_callback - INFO - Received callback for node 69c793c2be430e18c5d1606d INFO: 172.20.0.1:42362 - "POST /node/69c793c2be430e18c5d1606d HTTP/1.0" 202 Accepted 2026-03-28 08:58:55,625 - lava_callback - INFO - Starting async job submit for node 69c793c2be430e18c5d1606d 2026-03-28 08:58:55,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:55,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793c2be430e18c5d1606d HTTP/1.1" 200 None 2026-03-28 08:58:55,664 - lava_callback - DEBUG - Processing node 69c793c2be430e18c5d1606d: baseline-arm-pengutronix 2026-03-28 08:58:55,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:55,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:55,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c2be430e18c5d1606d/log.txt.gz 2026-03-28 08:58:55,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:56,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:56,170 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c2be430e18c5d1606d/lava-logs.yaml 2026-03-28 08:58:56,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:58:56,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:58:56,286 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c793c2be430e18c5d1606d/lava_callback.json.gz DEBUG: Setup failed for 69c793c2be430e18c5d1606d. Transitting job node result: incomplete -> fail 2026-03-28 08:58:56,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:56,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793c2be430e18c5d1606d HTTP/1.1" 200 None 2026-03-28 08:58:56,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:56,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 08:58:56,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:58:56,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793c2be430e18c5d1606d HTTP/1.1" 200 None 2026-03-28 08:58:56,486 - lava_callback - INFO - Completed processing callback for node 69c793c2be430e18c5d1606d INFO: 172.20.0.1:42376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42874 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:59:08,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:08,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47488 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:59:23,797 - lava_callback - INFO - Received callback for node 69c4f568f63ce05331e2b02f INFO: 172.20.0.1:33550 - "POST /node/69c4f568f63ce05331e2b02f HTTP/1.0" 202 Accepted 2026-03-28 08:59:23,803 - lava_callback - INFO - Starting async job submit for node 69c4f568f63ce05331e2b02f 2026-03-28 08:59:23,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:23,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f568f63ce05331e2b02f HTTP/1.1" 200 None 2026-03-28 08:59:23,825 - lava_callback - DEBUG - Processing node 69c4f568f63ce05331e2b02f: baseline-arm-clabbe 2026-03-28 08:59:23,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f568f63ce05331e2b02f 2026-03-28 08:59:23,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:23,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f568f63ce05331e2b02f HTTP/1.1" 200 None 2026-03-28 08:59:23,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:23,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-28 08:59:23,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:23,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f568f63ce05331e2b02f HTTP/1.1" 200 None 2026-03-28 08:59:23,883 - lava_callback - ERROR - Error processing callback for node 69c4f568f63ce05331e2b02f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33558 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:59:29,629 - lava_callback - INFO - Received callback for node 69c79729be430e18c5d163b8 INFO: 172.20.0.1:33560 - "POST /node/69c79729be430e18c5d163b8 HTTP/1.0" 202 Accepted 2026-03-28 08:59:29,637 - lava_callback - INFO - Starting async job submit for node 69c79729be430e18c5d163b8 2026-03-28 08:59:29,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:29,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79729be430e18c5d163b8 HTTP/1.1" 200 None 2026-03-28 08:59:29,686 - lava_callback - DEBUG - Processing node 69c79729be430e18c5d163b8: baseline-arm64 2026-03-28 08:59:30,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:59:30,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:59:30,244 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79729be430e18c5d163b8/log.txt.gz 2026-03-28 08:59:30,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:59:30,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:59:30,675 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79729be430e18c5d163b8/lava-logs.yaml 2026-03-28 08:59:30,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:59:30,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:59:30,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79729be430e18c5d163b8/lava_callback.json.gz DEBUG: Setup failed for 69c79729be430e18c5d163b8. Transitting job node result: incomplete -> fail 2026-03-28 08:59:30,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:30,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79729be430e18c5d163b8 HTTP/1.1" 200 None 2026-03-28 08:59:30,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:30,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 08:59:30,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:30,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79729be430e18c5d163b8 HTTP/1.1" 200 None 2026-03-28 08:59:31,058 - lava_callback - INFO - Completed processing callback for node 69c79729be430e18c5d163b8 INFO: 172.20.0.1:46252 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:59:38,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:59:38,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37644 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:00:04,033 - lava_callback - INFO - Received callback for node 69c4f58ef63ce05331e2b070 INFO: 172.20.0.1:53442 - "POST /node/69c4f58ef63ce05331e2b070 HTTP/1.0" 202 Accepted 2026-03-28 09:00:04,038 - lava_callback - INFO - Starting async job submit for node 69c4f58ef63ce05331e2b070 2026-03-28 09:00:04,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:04,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f58ef63ce05331e2b070 HTTP/1.1" 200 None 2026-03-28 09:00:04,055 - lava_callback - DEBUG - Processing node 69c4f58ef63ce05331e2b070: baseline-arm-clabbe 2026-03-28 09:00:04,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f58ef63ce05331e2b070 2026-03-28 09:00:04,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:04,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f58ef63ce05331e2b070 HTTP/1.1" 200 None 2026-03-28 09:00:04,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:04,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-28 09:00:04,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:04,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f58ef63ce05331e2b070 HTTP/1.1" 200 None 2026-03-28 09:00:04,116 - lava_callback - ERROR - Error processing callback for node 69c4f58ef63ce05331e2b070: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48600 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:00:43,841 - lava_callback - INFO - Received callback for node 69c4f5b3f63ce05331e2b098 INFO: 172.20.0.1:48608 - "POST /node/69c4f5b3f63ce05331e2b098 HTTP/1.0" 202 Accepted 2026-03-28 09:00:43,846 - lava_callback - INFO - Starting async job submit for node 69c4f5b3f63ce05331e2b098 2026-03-28 09:00:43,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:43,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5b3f63ce05331e2b098 HTTP/1.1" 200 None 2026-03-28 09:00:43,879 - lava_callback - DEBUG - Processing node 69c4f5b3f63ce05331e2b098: baseline-arm-clabbe 2026-03-28 09:00:43,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f5b3f63ce05331e2b098 2026-03-28 09:00:43,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:43,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5b3f63ce05331e2b098 HTTP/1.1" 200 None 2026-03-28 09:00:43,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:43,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-28 09:00:43,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:00:43,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f5b3f63ce05331e2b098 HTTP/1.1" 200 None 2026-03-28 09:00:43,934 - lava_callback - ERROR - Error processing callback for node 69c4f5b3f63ce05331e2b098: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39534 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:01:23,878 - lava_callback - INFO - Received callback for node 69c4f5d9f63ce05331e2b0b2 INFO: 172.20.0.1:39630 - "POST /node/69c4f5d9f63ce05331e2b0b2 HTTP/1.0" 202 Accepted 2026-03-28 09:01:23,883 - lava_callback - INFO - Starting async job submit for node 69c4f5d9f63ce05331e2b0b2 2026-03-28 09:01:23,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:01:23,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5d9f63ce05331e2b0b2 HTTP/1.1" 200 None 2026-03-28 09:01:23,900 - lava_callback - DEBUG - Processing node 69c4f5d9f63ce05331e2b0b2: baseline-arm-clabbe 2026-03-28 09:01:23,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f5d9f63ce05331e2b0b2 2026-03-28 09:01:23,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:01:23,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5d9f63ce05331e2b0b2 HTTP/1.1" 200 None 2026-03-28 09:01:23,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:01:23,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-28 09:01:23,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:01:23,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f5d9f63ce05331e2b0b2 HTTP/1.1" 200 None 2026-03-28 09:01:23,960 - lava_callback - ERROR - Error processing callback for node 69c4f5d9f63ce05331e2b0b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:02:03,914 - lava_callback - INFO - Received callback for node 69c4f5fef63ce05331e2b0c8 INFO: 172.20.0.1:60310 - "POST /node/69c4f5fef63ce05331e2b0c8 HTTP/1.0" 202 Accepted 2026-03-28 09:02:03,919 - lava_callback - INFO - Starting async job submit for node 69c4f5fef63ce05331e2b0c8 2026-03-28 09:02:03,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:03,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5fef63ce05331e2b0c8 HTTP/1.1" 200 None 2026-03-28 09:02:03,935 - lava_callback - DEBUG - Processing node 69c4f5fef63ce05331e2b0c8: baseline-arm-clabbe 2026-03-28 09:02:03,935 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f5fef63ce05331e2b0c8 2026-03-28 09:02:03,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:03,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f5fef63ce05331e2b0c8 HTTP/1.1" 200 None 2026-03-28 09:02:03,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:03,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-28 09:02:03,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:03,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f5fef63ce05331e2b0c8 HTTP/1.1" 200 None 2026-03-28 09:02:03,992 - lava_callback - ERROR - Error processing callback for node 69c4f5fef63ce05331e2b0c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43000 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:02:24,731 - lava_callback - INFO - Received callback for node 69c798e7be430e18c5d16651 INFO: 172.20.0.1:51940 - "POST /node/69c798e7be430e18c5d16651 HTTP/1.0" 202 Accepted 2026-03-28 09:02:24,737 - lava_callback - INFO - Starting async job submit for node 69c798e7be430e18c5d16651 2026-03-28 09:02:24,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:24,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c798e7be430e18c5d16651 HTTP/1.1" 200 None 2026-03-28 09:02:24,772 - lava_callback - DEBUG - Processing node 69c798e7be430e18c5d16651: baseline-arm-pengutronix 2026-03-28 09:02:24,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:02:25,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:02:25,081 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c798e7be430e18c5d16651/log.txt.gz 2026-03-28 09:02:25,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:02:25,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:02:25,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c798e7be430e18c5d16651/lava-logs.yaml 2026-03-28 09:02:25,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:02:25,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:02:25,503 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c798e7be430e18c5d16651/lava_callback.json.gz 2026-03-28 09:02:25,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:25,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c798e7be430e18c5d16651 HTTP/1.1" 200 None 2026-03-28 09:02:25,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:25,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 09:02:25,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:25,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c798e7be430e18c5d16651 HTTP/1.1" 200 None 2026-03-28 09:02:25,588 - lava_callback - INFO - Completed processing callback for node 69c798e7be430e18c5d16651 INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57808 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:02:38,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:02:38,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33158 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:03:13,250 - lava_callback - INFO - Received callback for node 69c4f64cf63ce05331e2b0e0 INFO: 172.20.0.1:38714 - "POST /node/69c4f64cf63ce05331e2b0e0 HTTP/1.0" 202 Accepted 2026-03-28 09:03:13,256 - lava_callback - INFO - Starting async job submit for node 69c4f64cf63ce05331e2b0e0 2026-03-28 09:03:13,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:03:13,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f64cf63ce05331e2b0e0 HTTP/1.1" 200 None 2026-03-28 09:03:13,273 - lava_callback - DEBUG - Processing node 69c4f64cf63ce05331e2b0e0: baseline-arm64-baylibre 2026-03-28 09:03:13,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f64cf63ce05331e2b0e0 2026-03-28 09:03:13,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:03:13,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f64cf63ce05331e2b0e0 HTTP/1.1" 200 None 2026-03-28 09:03:13,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:03:13,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:03:13,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:03:13,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f64cf63ce05331e2b0e0 HTTP/1.1" 200 None 2026-03-28 09:03:13,350 - lava_callback - ERROR - Error processing callback for node 69c4f64cf63ce05331e2b0e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42552 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:04:33,183 - lava_callback - INFO - Received callback for node 69c4f69bf63ce05331e2b0f0 INFO: 172.20.0.1:59818 - "POST /node/69c4f69bf63ce05331e2b0f0 HTTP/1.0" 202 Accepted 2026-03-28 09:04:33,189 - lava_callback - INFO - Starting async job submit for node 69c4f69bf63ce05331e2b0f0 2026-03-28 09:04:33,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:33,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f69bf63ce05331e2b0f0 HTTP/1.1" 200 None 2026-03-28 09:04:33,205 - lava_callback - DEBUG - Processing node 69c4f69bf63ce05331e2b0f0: baseline-arm64-baylibre 2026-03-28 09:04:33,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f69bf63ce05331e2b0f0 2026-03-28 09:04:33,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:33,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f69bf63ce05331e2b0f0 HTTP/1.1" 200 None 2026-03-28 09:04:33,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:33,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:04:33,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:33,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f69bf63ce05331e2b0f0 HTTP/1.1" 200 None 2026-03-28 09:04:33,281 - lava_callback - ERROR - Error processing callback for node 69c4f69bf63ce05331e2b0f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58666 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:04:44,762 - lava_callback - INFO - Received callback for node 69c79724be430e18c5d163ab INFO: 172.20.0.1:58680 - "POST /node/69c79724be430e18c5d163ab HTTP/1.0" 202 Accepted 2026-03-28 09:04:44,768 - lava_callback - INFO - Starting async job submit for node 69c79724be430e18c5d163ab 2026-03-28 09:04:44,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:44,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79724be430e18c5d163ab HTTP/1.1" 200 None 2026-03-28 09:04:44,807 - lava_callback - DEBUG - Processing node 69c79724be430e18c5d163ab: baseline-arm64 2026-03-28 09:04:45,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:04:45,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:04:45,442 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79724be430e18c5d163ab/log.txt.gz 2026-03-28 09:04:45,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:04:45,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:04:45,903 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79724be430e18c5d163ab/lava-logs.yaml 2026-03-28 09:04:45,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:04:46,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:04:46,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79724be430e18c5d163ab/lava_callback.json.gz 2026-03-28 09:04:46,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:46,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79724be430e18c5d163ab HTTP/1.1" 200 None 2026-03-28 09:04:46,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:46,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:04:46,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:04:46,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79724be430e18c5d163ab HTTP/1.1" 200 None 2026-03-28 09:04:46,210 - lava_callback - INFO - Completed processing callback for node 69c79724be430e18c5d163ab INFO: 172.20.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60064 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:05:08,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:05:08,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46782 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:05:53,202 - lava_callback - INFO - Received callback for node 69c4f6ecf63ce05331e2b102 INFO: 172.20.0.1:42970 - "POST /node/69c4f6ecf63ce05331e2b102 HTTP/1.0" 202 Accepted 2026-03-28 09:05:53,208 - lava_callback - INFO - Starting async job submit for node 69c4f6ecf63ce05331e2b102 2026-03-28 09:05:53,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:05:53,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f6ecf63ce05331e2b102 HTTP/1.1" 200 None 2026-03-28 09:05:53,224 - lava_callback - DEBUG - Processing node 69c4f6ecf63ce05331e2b102: baseline-arm64-baylibre 2026-03-28 09:05:53,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f6ecf63ce05331e2b102 2026-03-28 09:05:53,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:05:53,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f6ecf63ce05331e2b102 HTTP/1.1" 200 None 2026-03-28 09:05:53,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:05:53,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:05:53,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:05:53,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f6ecf63ce05331e2b102 HTTP/1.1" 200 None 2026-03-28 09:05:53,276 - lava_callback - ERROR - Error processing callback for node 69c4f6ecf63ce05331e2b102: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54566 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:06:12,217 - lava_callback - INFO - Received callback for node 69c79651be430e18c5d162d5 INFO: 172.20.0.1:40280 - "POST /node/69c79651be430e18c5d162d5 HTTP/1.0" 202 Accepted 2026-03-28 09:06:12,224 - lava_callback - INFO - Starting async job submit for node 69c79651be430e18c5d162d5 2026-03-28 09:06:12,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:06:12,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79651be430e18c5d162d5 HTTP/1.1" 200 None 2026-03-28 09:06:12,284 - lava_callback - DEBUG - Processing node 69c79651be430e18c5d162d5: baseline-x86 2026-03-28 09:06:13,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:06:13,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:06:13,227 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c79651be430e18c5d162d5/log.txt.gz 2026-03-28 09:06:13,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:06:14,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:06:14,068 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c79651be430e18c5d162d5/lava-logs.yaml 2026-03-28 09:06:14,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:06:14,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:06:14,284 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c79651be430e18c5d162d5/lava_callback.json.gz DEBUG: Setup failed for 69c79651be430e18c5d162d5. Transitting job node result: incomplete -> fail 2026-03-28 09:06:14,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:06:14,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79651be430e18c5d162d5 HTTP/1.1" 200 None 2026-03-28 09:06:14,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:06:14,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 09:06:14,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:06:14,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79651be430e18c5d162d5 HTTP/1.1" 200 None 2026-03-28 09:06:14,553 - lava_callback - INFO - Completed processing callback for node 69c79651be430e18c5d162d5 INFO: 172.20.0.1:40284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37544 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:06:38,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:06:38,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49322 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:07:13,328 - lava_callback - INFO - Received callback for node 69c4f73bf63ce05331e2b10f INFO: 172.20.0.1:56990 - "POST /node/69c4f73bf63ce05331e2b10f HTTP/1.0" 202 Accepted 2026-03-28 09:07:13,334 - lava_callback - INFO - Starting async job submit for node 69c4f73bf63ce05331e2b10f 2026-03-28 09:07:13,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:13,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f73bf63ce05331e2b10f HTTP/1.1" 200 None 2026-03-28 09:07:13,350 - lava_callback - DEBUG - Processing node 69c4f73bf63ce05331e2b10f: baseline-arm64-baylibre 2026-03-28 09:07:13,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f73bf63ce05331e2b10f 2026-03-28 09:07:13,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:13,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f73bf63ce05331e2b10f HTTP/1.1" 200 None 2026-03-28 09:07:13,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:13,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:07:13,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:13,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f73bf63ce05331e2b10f HTTP/1.1" 200 None 2026-03-28 09:07:13,421 - lava_callback - ERROR - Error processing callback for node 69c4f73bf63ce05331e2b10f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57016 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:07:17,943 - lava_callback - INFO - Received callback for node 69c79a38be430e18c5d166c8 INFO: 172.20.0.1:57024 - "POST /node/69c79a38be430e18c5d166c8 HTTP/1.0" 202 Accepted 2026-03-28 09:07:17,948 - lava_callback - INFO - Starting async job submit for node 69c79a38be430e18c5d166c8 2026-03-28 09:07:17,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:17,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79a38be430e18c5d166c8 HTTP/1.1" 200 None 2026-03-28 09:07:17,965 - lava_callback - DEBUG - Processing node 69c79a38be430e18c5d166c8: baseline-nfs-arm64 2026-03-28 09:07:17,966 - lava_callback - WARNING - Failed to upload log 2026-03-28 09:07:17,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:07:18,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:07:18,133 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79a38be430e18c5d166c8/lava-logs.yaml 2026-03-28 09:07:18,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:07:18,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:07:18,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79a38be430e18c5d166c8/lava_callback.json.gz 2026-03-28 09:07:18,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:18,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79a38be430e18c5d166c8 HTTP/1.1" 200 None 2026-03-28 09:07:18,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:18,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:07:18,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:18,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79a38be430e18c5d166c8 HTTP/1.1" 200 None 2026-03-28 09:07:18,350 - lava_callback - INFO - Completed processing callback for node 69c79a38be430e18c5d166c8 INFO: 172.20.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60368 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:07:38,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:07:38,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:08:33,466 - lava_callback - INFO - Received callback for node 69c4f78bf63ce05331e2b11d INFO: 172.20.0.1:45680 - "POST /node/69c4f78bf63ce05331e2b11d HTTP/1.0" 202 Accepted 2026-03-28 09:08:33,472 - lava_callback - INFO - Starting async job submit for node 69c4f78bf63ce05331e2b11d 2026-03-28 09:08:33,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:08:33,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f78bf63ce05331e2b11d HTTP/1.1" 200 None 2026-03-28 09:08:33,509 - lava_callback - DEBUG - Processing node 69c4f78bf63ce05331e2b11d: baseline-arm64-baylibre 2026-03-28 09:08:33,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f78bf63ce05331e2b11d 2026-03-28 09:08:33,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:08:33,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f78bf63ce05331e2b11d HTTP/1.1" 200 None 2026-03-28 09:08:33,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:08:33,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:08:33,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:08:33,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f78bf63ce05331e2b11d HTTP/1.1" 200 None 2026-03-28 09:08:33,560 - lava_callback - ERROR - Error processing callback for node 69c4f78bf63ce05331e2b11d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47732 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:10:48,191 - lava_callback - INFO - Received callback for node 69c7970bbe430e18c5d1636c INFO: 172.20.0.1:47744 - "POST /node/69c7970bbe430e18c5d1636c HTTP/1.0" 202 Accepted 2026-03-28 09:10:48,199 - lava_callback - INFO - Starting async job submit for node 69c7970bbe430e18c5d1636c 2026-03-28 09:10:48,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:48,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7970bbe430e18c5d1636c HTTP/1.1" 200 None 2026-03-28 09:10:48,279 - lava_callback - DEBUG - Processing node 69c7970bbe430e18c5d1636c: ltp-smoke 2026-03-28 09:10:48,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:10:49,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:10:49,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7970bbe430e18c5d1636c/log.txt.gz 2026-03-28 09:10:49,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:10:49,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:10:49,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7970bbe430e18c5d1636c/lava-logs.yaml 2026-03-28 09:10:49,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:10:50,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:10:50,031 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7970bbe430e18c5d1636c/lava_callback.json.gz 2026-03-28 09:10:50,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:50,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7970bbe430e18c5d1636c HTTP/1.1" 200 None 2026-03-28 09:10:50,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:50,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb1be430e18c5d159e5 HTTP/1.1" 200 None 2026-03-28 09:10:50,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:50,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7970bbe430e18c5d1636c HTTP/1.1" 200 None 2026-03-28 09:10:50,409 - lava_callback - INFO - Completed processing callback for node 69c7970bbe430e18c5d1636c INFO: 172.20.0.1:38804 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:10:58,998 - lava_callback - INFO - Received callback for node 69c79af2be430e18c5d166ef INFO: 172.20.0.1:38818 - "POST /node/69c79af2be430e18c5d166ef HTTP/1.0" 202 Accepted 2026-03-28 09:10:59,009 - lava_callback - INFO - Starting async job submit for node 69c79af2be430e18c5d166ef 2026-03-28 09:10:59,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:59,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79af2be430e18c5d166ef HTTP/1.1" 200 None 2026-03-28 09:10:59,072 - lava_callback - DEBUG - Processing node 69c79af2be430e18c5d166ef: baseline-arm64 2026-03-28 09:10:59,206 - lava_callback - WARNING - Failed to upload log 2026-03-28 09:10:59,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:10:59,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:10:59,668 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79af2be430e18c5d166ef/lava-logs.yaml 2026-03-28 09:10:59,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:10:59,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:10:59,815 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79af2be430e18c5d166ef/lava_callback.json.gz 2026-03-28 09:10:59,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:59,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79af2be430e18c5d166ef HTTP/1.1" 200 None 2026-03-28 09:10:59,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:59,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:10:59,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:10:59,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79af2be430e18c5d166ef HTTP/1.1" 200 None 2026-03-28 09:11:00,000 - lava_callback - INFO - Completed processing callback for node 69c79af2be430e18c5d166ef INFO: 172.20.0.1:49266 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:11:08,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:11:08,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56260 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:11:44,403 - lava_callback - INFO - Received callback for node 69c4f849f63ce05331e2b139 INFO: 172.20.0.1:56264 - "POST /node/69c4f849f63ce05331e2b139 HTTP/1.0" 202 Accepted 2026-03-28 09:11:44,409 - lava_callback - INFO - Starting async job submit for node 69c4f849f63ce05331e2b139 2026-03-28 09:11:44,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:11:44,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f849f63ce05331e2b139 HTTP/1.1" 200 None 2026-03-28 09:11:44,425 - lava_callback - DEBUG - Processing node 69c4f849f63ce05331e2b139: baseline-arm64-clabbe 2026-03-28 09:11:44,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f849f63ce05331e2b139 2026-03-28 09:11:44,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:11:44,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f849f63ce05331e2b139 HTTP/1.1" 200 None 2026-03-28 09:11:44,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:11:44,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:11:44,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:11:44,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f849f63ce05331e2b139 HTTP/1.1" 200 None 2026-03-28 09:11:44,490 - lava_callback - ERROR - Error processing callback for node 69c4f849f63ce05331e2b139: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59916 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:12:34,684 - lava_callback - INFO - Received callback for node 69c79b43be430e18c5d16701 INFO: 172.20.0.1:37442 - "POST /node/69c79b43be430e18c5d16701 HTTP/1.0" 202 Accepted 2026-03-28 09:12:34,691 - lava_callback - INFO - Starting async job submit for node 69c79b43be430e18c5d16701 2026-03-28 09:12:34,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:12:34,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79b43be430e18c5d16701 HTTP/1.1" 200 None 2026-03-28 09:12:34,748 - lava_callback - DEBUG - Processing node 69c79b43be430e18c5d16701: baseline-arm64 2026-03-28 09:12:34,882 - lava_callback - WARNING - Failed to upload log 2026-03-28 09:12:34,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:12:35,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:12:35,128 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79b43be430e18c5d16701/lava-logs.yaml 2026-03-28 09:12:35,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:12:35,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:12:35,247 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79b43be430e18c5d16701/lava_callback.json.gz 2026-03-28 09:12:35,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:12:35,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79b43be430e18c5d16701 HTTP/1.1" 200 None 2026-03-28 09:12:35,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:12:35,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:12:35,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:12:35,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79b43be430e18c5d16701 HTTP/1.1" 200 None 2026-03-28 09:12:35,421 - lava_callback - INFO - Completed processing callback for node 69c79b43be430e18c5d16701 INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:12:38,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:12:38,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50276 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:13:04,459 - lava_callback - INFO - Received callback for node 69c4f899f63ce05331e2b14d INFO: 172.20.0.1:60624 - "POST /node/69c4f899f63ce05331e2b14d HTTP/1.0" 202 Accepted 2026-03-28 09:13:04,465 - lava_callback - INFO - Starting async job submit for node 69c4f899f63ce05331e2b14d 2026-03-28 09:13:04,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:04,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f899f63ce05331e2b14d HTTP/1.1" 200 None 2026-03-28 09:13:04,490 - lava_callback - DEBUG - Processing node 69c4f899f63ce05331e2b14d: baseline-arm64-clabbe 2026-03-28 09:13:04,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f899f63ce05331e2b14d 2026-03-28 09:13:04,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:04,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f899f63ce05331e2b14d HTTP/1.1" 200 None 2026-03-28 09:13:04,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:04,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:13:04,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:04,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f899f63ce05331e2b14d HTTP/1.1" 200 None 2026-03-28 09:13:04,553 - lava_callback - ERROR - Error processing callback for node 69c4f899f63ce05331e2b14d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42332 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:13:36,369 - lava_callback - INFO - Received callback for node 69c793edbe430e18c5d160fd INFO: 172.20.0.1:44510 - "POST /node/69c793edbe430e18c5d160fd HTTP/1.0" 202 Accepted 2026-03-28 09:13:36,417 - lava_callback - INFO - Starting async job submit for node 69c793edbe430e18c5d160fd 2026-03-28 09:13:36,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:36,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793edbe430e18c5d160fd HTTP/1.1" 200 None 2026-03-28 09:13:36,458 - lava_callback - DEBUG - Processing node 69c793edbe430e18c5d160fd: ltp-watchqueue INFO: 172.20.0.1:44524 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:13:36,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:36,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:36,696 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c793edbe430e18c5d160fd/log.txt.gz 2026-03-28 09:13:36,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:36,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:36,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c793edbe430e18c5d160fd/lava-logs.yaml 2026-03-28 09:13:36,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:37,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:37,086 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c793edbe430e18c5d160fd/lava_callback.json.gz 2026-03-28 09:13:37,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:37,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c793edbe430e18c5d160fd HTTP/1.1" 200 None 2026-03-28 09:13:37,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:37,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 09:13:37,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:37,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c793edbe430e18c5d160fd HTTP/1.1" 200 None 2026-03-28 09:13:37,227 - lava_callback - INFO - Completed processing callback for node 69c793edbe430e18c5d160fd 2026-03-28 09:13:38,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:38,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 09:13:39,011 - lava_callback - INFO - Received callback for node 69c79b95be430e18c5d16714 INFO: 172.20.0.1:44536 - "POST /node/69c79b95be430e18c5d16714 HTTP/1.0" 202 Accepted 2026-03-28 09:13:39,017 - lava_callback - INFO - Starting async job submit for node 69c79b95be430e18c5d16714 2026-03-28 09:13:39,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:39,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79b95be430e18c5d16714 HTTP/1.1" 200 None 2026-03-28 09:13:39,054 - lava_callback - DEBUG - Processing node 69c79b95be430e18c5d16714: baseline-arm64 2026-03-28 09:13:39,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:39,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:39,572 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79b95be430e18c5d16714/log.txt.gz 2026-03-28 09:13:39,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:40,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:40,034 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79b95be430e18c5d16714/lava-logs.yaml 2026-03-28 09:13:40,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:40,207 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:40,209 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79b95be430e18c5d16714/lava_callback.json.gz 2026-03-28 09:13:40,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:40,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79b95be430e18c5d16714 HTTP/1.1" 200 None 2026-03-28 09:13:40,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:40,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:13:40,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:40,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79b95be430e18c5d16714 HTTP/1.1" 200 None 2026-03-28 09:13:40,290 - lava_callback - INFO - Completed processing callback for node 69c79b95be430e18c5d16714 INFO: 172.20.0.1:40370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40384 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:13:52,846 - lava_callback - INFO - Received callback for node 69c7972ebe430e18c5d163e4 INFO: 172.20.0.1:38008 - "POST /node/69c7972ebe430e18c5d163e4 HTTP/1.0" 202 Accepted 2026-03-28 09:13:52,855 - lava_callback - INFO - Starting async job submit for node 69c7972ebe430e18c5d163e4 2026-03-28 09:13:52,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:52,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972ebe430e18c5d163e4 HTTP/1.1" 200 None 2026-03-28 09:13:52,921 - lava_callback - DEBUG - Processing node 69c7972ebe430e18c5d163e4: baseline-nfs-arm64 2026-03-28 09:13:53,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:54,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:54,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972ebe430e18c5d163e4/log.txt.gz 2026-03-28 09:13:54,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:54,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:54,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972ebe430e18c5d163e4/lava-logs.yaml 2026-03-28 09:13:54,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:13:55,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:13:55,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7972ebe430e18c5d163e4/lava_callback.json.gz DEBUG: Setup failed for 69c7972ebe430e18c5d163e4. Transitting job node result: incomplete -> fail 2026-03-28 09:13:55,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:55,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7972ebe430e18c5d163e4 HTTP/1.1" 200 None 2026-03-28 09:13:55,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:55,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:13:55,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:13:55,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7972ebe430e18c5d163e4 HTTP/1.1" 200 None 2026-03-28 09:13:55,338 - lava_callback - INFO - Completed processing callback for node 69c7972ebe430e18c5d163e4 INFO: 172.20.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56160 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:14:08,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:14:08,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:14:24,482 - lava_callback - INFO - Received callback for node 69c4f8eaf63ce05331e2b156 INFO: 172.20.0.1:33116 - "POST /node/69c4f8eaf63ce05331e2b156 HTTP/1.0" 202 Accepted 2026-03-28 09:14:24,487 - lava_callback - INFO - Starting async job submit for node 69c4f8eaf63ce05331e2b156 2026-03-28 09:14:24,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:14:24,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f8eaf63ce05331e2b156 HTTP/1.1" 200 None 2026-03-28 09:14:24,503 - lava_callback - DEBUG - Processing node 69c4f8eaf63ce05331e2b156: baseline-arm64-clabbe 2026-03-28 09:14:24,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f8eaf63ce05331e2b156 2026-03-28 09:14:24,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:14:24,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f8eaf63ce05331e2b156 HTTP/1.1" 200 None 2026-03-28 09:14:24,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:14:24,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:14:24,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:14:24,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f8eaf63ce05331e2b156 HTTP/1.1" 200 None 2026-03-28 09:14:24,563 - lava_callback - ERROR - Error processing callback for node 69c4f8eaf63ce05331e2b156: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41940 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:15:10,027 - lava_callback - INFO - Received callback for node 69c79be5be430e18c5d16732 INFO: 172.20.0.1:54644 - "POST /node/69c79be5be430e18c5d16732 HTTP/1.0" 202 Accepted 2026-03-28 09:15:10,033 - lava_callback - INFO - Starting async job submit for node 69c79be5be430e18c5d16732 2026-03-28 09:15:10,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:10,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79be5be430e18c5d16732 HTTP/1.1" 200 None 2026-03-28 09:15:10,073 - lava_callback - DEBUG - Processing node 69c79be5be430e18c5d16732: baseline-arm64 2026-03-28 09:15:10,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:15:10,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:15:10,564 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79be5be430e18c5d16732/log.txt.gz 2026-03-28 09:15:10,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:15:10,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:15:10,987 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79be5be430e18c5d16732/lava-logs.yaml 2026-03-28 09:15:10,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:15:11,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:15:11,342 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79be5be430e18c5d16732/lava_callback.json.gz 2026-03-28 09:15:11,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:11,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79be5be430e18c5d16732 HTTP/1.1" 200 None 2026-03-28 09:15:11,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:11,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:15:11,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:11,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79be5be430e18c5d16732 HTTP/1.1" 200 None 2026-03-28 09:15:11,524 - lava_callback - INFO - Completed processing callback for node 69c79be5be430e18c5d16732 INFO: 172.20.0.1:54658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35472 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:15:38,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:38,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52720 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:15:44,515 - lava_callback - INFO - Received callback for node 69c4f93cf63ce05331e2b162 INFO: 172.20.0.1:52736 - "POST /node/69c4f93cf63ce05331e2b162 HTTP/1.0" 202 Accepted 2026-03-28 09:15:44,520 - lava_callback - INFO - Starting async job submit for node 69c4f93cf63ce05331e2b162 2026-03-28 09:15:44,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:44,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f93cf63ce05331e2b162 HTTP/1.1" 200 None 2026-03-28 09:15:44,535 - lava_callback - DEBUG - Processing node 69c4f93cf63ce05331e2b162: baseline-arm64-clabbe 2026-03-28 09:15:44,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f93cf63ce05331e2b162 2026-03-28 09:15:44,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:44,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f93cf63ce05331e2b162 HTTP/1.1" 200 None 2026-03-28 09:15:44,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:44,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:15:44,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:15:44,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f93cf63ce05331e2b162 HTTP/1.1" 200 None 2026-03-28 09:15:44,586 - lava_callback - ERROR - Error processing callback for node 69c4f93cf63ce05331e2b162: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56208 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:17:04,583 - lava_callback - INFO - Received callback for node 69c4f98df63ce05331e2b171 INFO: 172.20.0.1:37790 - "POST /node/69c4f98df63ce05331e2b171 HTTP/1.0" 202 Accepted 2026-03-28 09:17:04,593 - lava_callback - INFO - Starting async job submit for node 69c4f98df63ce05331e2b171 2026-03-28 09:17:04,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:04,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f98df63ce05331e2b171 HTTP/1.1" 200 None 2026-03-28 09:17:04,627 - lava_callback - DEBUG - Processing node 69c4f98df63ce05331e2b171: baseline-arm64-clabbe 2026-03-28 09:17:04,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f98df63ce05331e2b171 2026-03-28 09:17:04,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:04,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f98df63ce05331e2b171 HTTP/1.1" 200 None 2026-03-28 09:17:04,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:04,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:17:04,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:04,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f98df63ce05331e2b171 HTTP/1.1" 200 None 2026-03-28 09:17:04,692 - lava_callback - ERROR - Error processing callback for node 69c4f98df63ce05331e2b171: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49424 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:17:23,253 - lava_callback - INFO - Received callback for node 69c79734be430e18c5d163f8 INFO: 172.20.0.1:39588 - "POST /node/69c79734be430e18c5d163f8 HTTP/1.0" 202 Accepted 2026-03-28 09:17:23,260 - lava_callback - INFO - Starting async job submit for node 69c79734be430e18c5d163f8 2026-03-28 09:17:23,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:23,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79734be430e18c5d163f8 HTTP/1.1" 200 None 2026-03-28 09:17:23,300 - lava_callback - DEBUG - Processing node 69c79734be430e18c5d163f8: baseline-nfs-arm64 2026-03-28 09:17:23,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:17:23,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:17:23,988 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79734be430e18c5d163f8/log.txt.gz 2026-03-28 09:17:24,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:17:24,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:17:24,499 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79734be430e18c5d163f8/lava-logs.yaml 2026-03-28 09:17:24,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:17:24,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:17:24,687 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c79734be430e18c5d163f8/lava_callback.json.gz 2026-03-28 09:17:24,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:24,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79734be430e18c5d163f8 HTTP/1.1" 200 None 2026-03-28 09:17:24,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:24,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:17:24,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:24,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79734be430e18c5d163f8 HTTP/1.1" 200 None 2026-03-28 09:17:24,839 - lava_callback - INFO - Completed processing callback for node 69c79734be430e18c5d163f8 INFO: 172.20.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34032 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:17:38,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:38,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 09:17:41,999 - lava_callback - INFO - Received callback for node 69c79c56be430e18c5d16758 INFO: 172.20.0.1:55454 - "POST /node/69c79c56be430e18c5d16758 HTTP/1.0" 202 Accepted 2026-03-28 09:17:42,005 - lava_callback - INFO - Starting async job submit for node 69c79c56be430e18c5d16758 2026-03-28 09:17:42,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:42,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79c56be430e18c5d16758 HTTP/1.1" 200 None 2026-03-28 09:17:42,041 - lava_callback - DEBUG - Processing node 69c79c56be430e18c5d16758: baseline-arm64 2026-03-28 09:17:42,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:17:42,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:17:42,437 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79c56be430e18c5d16758/log.txt.gz 2026-03-28 09:17:42,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:17:42,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:17:42,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79c56be430e18c5d16758/lava-logs.yaml 2026-03-28 09:17:42,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:17:42,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:17:42,926 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c79c56be430e18c5d16758/lava_callback.json.gz 2026-03-28 09:17:42,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:42,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79c56be430e18c5d16758 HTTP/1.1" 200 None 2026-03-28 09:17:42,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:42,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:17:42,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:17:43,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79c56be430e18c5d16758 HTTP/1.1" 200 None 2026-03-28 09:17:43,007 - lava_callback - INFO - Completed processing callback for node 69c79c56be430e18c5d16758 INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44540 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:18:08,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:18:08,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44942 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:18:24,625 - lava_callback - INFO - Received callback for node 69c4f9ddf63ce05331e2b17d INFO: 172.20.0.1:33764 - "POST /node/69c4f9ddf63ce05331e2b17d HTTP/1.0" 202 Accepted 2026-03-28 09:18:24,630 - lava_callback - INFO - Starting async job submit for node 69c4f9ddf63ce05331e2b17d 2026-03-28 09:18:24,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:18:24,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f9ddf63ce05331e2b17d HTTP/1.1" 200 None 2026-03-28 09:18:24,647 - lava_callback - DEBUG - Processing node 69c4f9ddf63ce05331e2b17d: baseline-arm64-clabbe 2026-03-28 09:18:24,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4f9ddf63ce05331e2b17d 2026-03-28 09:18:24,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:18:24,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4f9ddf63ce05331e2b17d HTTP/1.1" 200 None 2026-03-28 09:18:24,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:18:24,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 09:18:24,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:18:24,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4f9ddf63ce05331e2b17d HTTP/1.1" 200 None 2026-03-28 09:18:24,710 - lava_callback - ERROR - Error processing callback for node 69c4f9ddf63ce05331e2b17d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46380 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:32:23,914 - lava_callback - INFO - Received callback for node 69c79c32be430e18c5d16742 INFO: 172.20.0.1:36588 - "POST /node/69c79c32be430e18c5d16742 HTTP/1.0" 202 Accepted 2026-03-28 09:32:23,920 - lava_callback - INFO - Starting async job submit for node 69c79c32be430e18c5d16742 2026-03-28 09:32:23,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:32:23,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79c32be430e18c5d16742 HTTP/1.1" 200 None 2026-03-28 09:32:23,989 - lava_callback - DEBUG - Processing node 69c79c32be430e18c5d16742: baseline-arm 2026-03-28 09:32:24,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:32:24,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:32:24,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c79c32be430e18c5d16742/log.txt.gz 2026-03-28 09:32:25,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:32:25,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:32:25,356 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c79c32be430e18c5d16742/lava-logs.yaml 2026-03-28 09:32:25,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:32:25,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:32:25,563 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c79c32be430e18c5d16742/lava_callback.json.gz DEBUG: Setup failed for 69c79c32be430e18c5d16742. Transitting job node result: incomplete -> fail 2026-03-28 09:32:25,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:32:25,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79c32be430e18c5d16742 HTTP/1.1" 200 None 2026-03-28 09:32:25,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:32:25,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 09:32:25,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:32:25,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79c32be430e18c5d16742 HTTP/1.1" 200 None 2026-03-28 09:32:25,844 - lava_callback - INFO - Completed processing callback for node 69c79c32be430e18c5d16742 INFO: 172.20.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36896 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:32:38,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:32:38,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35020 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:33:34,840 - lava_callback - INFO - Received callback for node 69c4fd59f63ce05331e2b233 INFO: 172.20.0.1:59526 - "POST /node/69c4fd59f63ce05331e2b233 HTTP/1.0" 202 Accepted 2026-03-28 09:33:34,845 - lava_callback - INFO - Starting async job submit for node 69c4fd59f63ce05331e2b233 2026-03-28 09:33:34,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:33:34,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fd59f63ce05331e2b233 HTTP/1.1" 200 None 2026-03-28 09:33:34,861 - lava_callback - DEBUG - Processing node 69c4fd59f63ce05331e2b233: baseline-arm64-baylibre 2026-03-28 09:33:34,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4fd59f63ce05331e2b233 2026-03-28 09:33:34,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:33:34,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fd59f63ce05331e2b233 HTTP/1.1" 200 None 2026-03-28 09:33:34,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:33:34,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 09:33:34,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:33:34,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fd59f63ce05331e2b233 HTTP/1.1" 200 None 2026-03-28 09:33:34,920 - lava_callback - ERROR - Error processing callback for node 69c4fd59f63ce05331e2b233: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37816 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:35:25,365 - lava_callback - INFO - Received callback for node 69c4fdd5f63ce05331e2b24e INFO: 172.20.0.1:46454 - "POST /node/69c4fdd5f63ce05331e2b24e HTTP/1.0" 202 Accepted 2026-03-28 09:35:25,371 - lava_callback - INFO - Starting async job submit for node 69c4fdd5f63ce05331e2b24e 2026-03-28 09:35:25,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:35:25,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fdd5f63ce05331e2b24e HTTP/1.1" 200 None 2026-03-28 09:35:25,387 - lava_callback - DEBUG - Processing node 69c4fdd5f63ce05331e2b24e: baseline-arm64-clabbe 2026-03-28 09:35:25,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4fdd5f63ce05331e2b24e 2026-03-28 09:35:25,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:35:25,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fdd5f63ce05331e2b24e HTTP/1.1" 200 None 2026-03-28 09:35:25,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:35:25,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 09:35:25,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:35:25,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fdd5f63ce05331e2b24e HTTP/1.1" 200 None 2026-03-28 09:35:25,476 - lava_callback - ERROR - Error processing callback for node 69c4fdd5f63ce05331e2b24e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54808 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:36:45,379 - lava_callback - INFO - Received callback for node 69c4fe24f63ce05331e2b25d INFO: 172.20.0.1:54820 - "POST /node/69c4fe24f63ce05331e2b25d HTTP/1.0" 202 Accepted 2026-03-28 09:36:45,385 - lava_callback - INFO - Starting async job submit for node 69c4fe24f63ce05331e2b25d 2026-03-28 09:36:45,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:36:45,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fe24f63ce05331e2b25d HTTP/1.1" 200 None 2026-03-28 09:36:45,400 - lava_callback - DEBUG - Processing node 69c4fe24f63ce05331e2b25d: baseline-arm64-clabbe 2026-03-28 09:36:45,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4fe24f63ce05331e2b25d 2026-03-28 09:36:45,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:36:45,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fe24f63ce05331e2b25d HTTP/1.1" 200 None 2026-03-28 09:36:45,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:36:45,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 09:36:45,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:36:45,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fe24f63ce05331e2b25d HTTP/1.1" 200 None 2026-03-28 09:36:45,464 - lava_callback - ERROR - Error processing callback for node 69c4fe24f63ce05331e2b25d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42230 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:38:05,451 - lava_callback - INFO - Received callback for node 69c4fe74f63ce05331e2b268 INFO: 172.20.0.1:43576 - "POST /node/69c4fe74f63ce05331e2b268 HTTP/1.0" 202 Accepted 2026-03-28 09:38:05,457 - lava_callback - INFO - Starting async job submit for node 69c4fe74f63ce05331e2b268 2026-03-28 09:38:05,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:38:05,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fe74f63ce05331e2b268 HTTP/1.1" 200 None 2026-03-28 09:38:05,480 - lava_callback - DEBUG - Processing node 69c4fe74f63ce05331e2b268: baseline-arm64-clabbe 2026-03-28 09:38:05,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4fe74f63ce05331e2b268 2026-03-28 09:38:05,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:38:05,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4fe74f63ce05331e2b268 HTTP/1.1" 200 None 2026-03-28 09:38:05,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:38:05,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 09:38:05,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:38:05,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4fe74f63ce05331e2b268 HTTP/1.1" 200 None 2026-03-28 09:38:05,533 - lava_callback - ERROR - Error processing callback for node 69c4fe74f63ce05331e2b268: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34066 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:41:25,573 - lava_callback - INFO - Received callback for node 69c4ff3af63ce05331e2b280 INFO: 172.20.0.1:41190 - "POST /node/69c4ff3af63ce05331e2b280 HTTP/1.0" 202 Accepted 2026-03-28 09:41:25,578 - lava_callback - INFO - Starting async job submit for node 69c4ff3af63ce05331e2b280 2026-03-28 09:41:25,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:41:25,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ff3af63ce05331e2b280 HTTP/1.1" 200 None 2026-03-28 09:41:25,594 - lava_callback - DEBUG - Processing node 69c4ff3af63ce05331e2b280: baseline-arm64-clabbe 2026-03-28 09:41:25,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ff3af63ce05331e2b280 2026-03-28 09:41:25,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:41:25,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ff3af63ce05331e2b280 HTTP/1.1" 200 None 2026-03-28 09:41:25,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:41:25,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 09:41:25,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:41:25,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ff3af63ce05331e2b280 HTTP/1.1" 200 None 2026-03-28 09:41:25,645 - lava_callback - ERROR - Error processing callback for node 69c4ff3af63ce05331e2b280: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51936 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:42:45,125 - lava_callback - INFO - Received callback for node 69c7a0ccbe430e18c5d167fc INFO: 172.20.0.1:51938 - "POST /node/69c7a0ccbe430e18c5d167fc HTTP/1.0" 202 Accepted 2026-03-28 09:42:45,132 - lava_callback - INFO - Starting async job submit for node 69c7a0ccbe430e18c5d167fc 2026-03-28 09:42:45,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:45,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a0ccbe430e18c5d167fc HTTP/1.1" 200 None 2026-03-28 09:42:45,183 - lava_callback - DEBUG - Processing node 69c7a0ccbe430e18c5d167fc: baseline-arm64 2026-03-28 09:42:45,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:42:45,661 - lava_callback - INFO - Received callback for node 69c4ff89f63ce05331e2b28a INFO: 172.20.0.1:51950 - "POST /node/69c4ff89f63ce05331e2b28a HTTP/1.0" 202 Accepted 2026-03-28 09:42:45,667 - lava_callback - INFO - Starting async job submit for node 69c4ff89f63ce05331e2b28a 2026-03-28 09:42:45,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:45,681 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:42:45,683 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a0ccbe430e18c5d167fc/log.txt.gz 2026-03-28 09:42:45,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ff89f63ce05331e2b28a HTTP/1.1" 200 None 2026-03-28 09:42:45,724 - lava_callback - DEBUG - Processing node 69c4ff89f63ce05331e2b28a: baseline-arm64-clabbe 2026-03-28 09:42:45,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ff89f63ce05331e2b28a 2026-03-28 09:42:45,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:45,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ff89f63ce05331e2b28a HTTP/1.1" 200 None 2026-03-28 09:42:45,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:45,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 09:42:45,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:45,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:42:45,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ff89f63ce05331e2b28a HTTP/1.1" 200 None 2026-03-28 09:42:45,965 - lava_callback - ERROR - Error processing callback for node 69c4ff89f63ce05331e2b28a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 09:42:46,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:42:46,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a0ccbe430e18c5d167fc/lava-logs.yaml 2026-03-28 09:42:46,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:42:46,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:42:46,462 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a0ccbe430e18c5d167fc/lava_callback.json.gz INFO: 172.20.0.1:51960 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 69c7a0ccbe430e18c5d167fc. Transitting job node result: incomplete -> fail 2026-03-28 09:42:46,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:46,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a0ccbe430e18c5d167fc HTTP/1.1" 200 None 2026-03-28 09:42:46,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:46,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:42:46,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:42:46,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a0ccbe430e18c5d167fc HTTP/1.1" 200 None 2026-03-28 09:42:46,675 - lava_callback - INFO - Completed processing callback for node 69c7a0ccbe430e18c5d167fc INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39884 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:43:08,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:43:08,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33532 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:44:05,926 - lava_callback - INFO - Received callback for node 69c4ffd8f63ce05331e2b28f INFO: 172.20.0.1:44394 - "POST /node/69c4ffd8f63ce05331e2b28f HTTP/1.0" 202 Accepted 2026-03-28 09:44:05,932 - lava_callback - INFO - Starting async job submit for node 69c4ffd8f63ce05331e2b28f 2026-03-28 09:44:05,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:44:05,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ffd8f63ce05331e2b28f HTTP/1.1" 200 None 2026-03-28 09:44:05,950 - lava_callback - DEBUG - Processing node 69c4ffd8f63ce05331e2b28f: baseline-arm64-clabbe 2026-03-28 09:44:05,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ffd8f63ce05331e2b28f 2026-03-28 09:44:05,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:44:05,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ffd8f63ce05331e2b28f HTTP/1.1" 200 None 2026-03-28 09:44:05,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:44:05,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 09:44:05,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:44:06,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ffd8f63ce05331e2b28f HTTP/1.1" 200 None 2026-03-28 09:44:06,002 - lava_callback - ERROR - Error processing callback for node 69c4ffd8f63ce05331e2b28f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51000 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 09:45:15,621 - lava_callback - INFO - Received callback for node 69c50021f63ce05331e2b2a1 INFO: 172.20.0.1:51002 - "POST /node/69c50021f63ce05331e2b2a1 HTTP/1.0" 202 Accepted 2026-03-28 09:45:15,627 - lava_callback - INFO - Starting async job submit for node 69c50021f63ce05331e2b2a1 2026-03-28 09:45:15,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:45:15,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50021f63ce05331e2b2a1 HTTP/1.1" 200 None 2026-03-28 09:45:15,642 - lava_callback - DEBUG - Processing node 69c50021f63ce05331e2b2a1: baseline-arm64-baylibre 2026-03-28 09:45:15,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50021f63ce05331e2b2a1 2026-03-28 09:45:15,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:45:15,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50021f63ce05331e2b2a1 HTTP/1.1" 200 None 2026-03-28 09:45:15,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:45:15,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-28 09:45:15,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:45:15,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50021f63ce05331e2b2a1 HTTP/1.1" 200 None 2026-03-28 09:45:15,701 - lava_callback - ERROR - Error processing callback for node 69c50021f63ce05331e2b2a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52470 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:46:45,790 - lava_callback - INFO - Received callback for node 69c50077f63ce05331e2b2aa INFO: 172.20.0.1:52474 - "POST /node/69c50077f63ce05331e2b2aa HTTP/1.0" 202 Accepted 2026-03-28 09:46:45,795 - lava_callback - INFO - Starting async job submit for node 69c50077f63ce05331e2b2aa 2026-03-28 09:46:45,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:46:45,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50077f63ce05331e2b2aa HTTP/1.1" 200 None 2026-03-28 09:46:45,810 - lava_callback - DEBUG - Processing node 69c50077f63ce05331e2b2aa: baseline-arm64-clabbe 2026-03-28 09:46:45,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50077f63ce05331e2b2aa 2026-03-28 09:46:45,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:46:45,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50077f63ce05331e2b2aa HTTP/1.1" 200 None 2026-03-28 09:46:45,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:46:45,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-28 09:46:45,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:46:45,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50077f63ce05331e2b2aa HTTP/1.1" 200 None 2026-03-28 09:46:45,887 - lava_callback - ERROR - Error processing callback for node 69c50077f63ce05331e2b2aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:47:05,428 - lava_callback - INFO - Received callback for node 69c7a38abe430e18c5d16868 INFO: 172.20.0.1:51364 - "POST /node/69c7a38abe430e18c5d16868 HTTP/1.0" 202 Accepted 2026-03-28 09:47:05,433 - lava_callback - INFO - Starting async job submit for node 69c7a38abe430e18c5d16868 2026-03-28 09:47:05,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:47:05,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a38abe430e18c5d16868 HTTP/1.1" 200 None 2026-03-28 09:47:05,450 - lava_callback - DEBUG - Processing node 69c7a38abe430e18c5d16868: baseline-nfs-arm64 2026-03-28 09:47:05,452 - lava_callback - WARNING - Failed to upload log 2026-03-28 09:47:05,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:47:05,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:47:05,779 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a38abe430e18c5d16868/lava-logs.yaml 2026-03-28 09:47:05,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:47:05,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:47:05,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a38abe430e18c5d16868/lava_callback.json.gz 2026-03-28 09:47:05,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:47:05,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a38abe430e18c5d16868 HTTP/1.1" 200 None 2026-03-28 09:47:05,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:47:05,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:47:05,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:47:05,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a38abe430e18c5d16868 HTTP/1.1" 200 None 2026-03-28 09:47:05,999 - lava_callback - INFO - Completed processing callback for node 69c7a38abe430e18c5d16868 INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:47:08,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:47:08,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:48:05,843 - lava_callback - INFO - Received callback for node 69c500c7f63ce05331e2b2bc INFO: 172.20.0.1:38648 - "POST /node/69c500c7f63ce05331e2b2bc HTTP/1.0" 202 Accepted 2026-03-28 09:48:05,849 - lava_callback - INFO - Starting async job submit for node 69c500c7f63ce05331e2b2bc 2026-03-28 09:48:05,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:48:05,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c500c7f63ce05331e2b2bc HTTP/1.1" 200 None 2026-03-28 09:48:05,864 - lava_callback - DEBUG - Processing node 69c500c7f63ce05331e2b2bc: baseline-arm64-clabbe 2026-03-28 09:48:05,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c500c7f63ce05331e2b2bc 2026-03-28 09:48:05,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:48:05,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c500c7f63ce05331e2b2bc HTTP/1.1" 200 None 2026-03-28 09:48:05,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:48:05,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-28 09:48:05,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:48:05,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c500c7f63ce05331e2b2bc HTTP/1.1" 200 None 2026-03-28 09:48:05,918 - lava_callback - ERROR - Error processing callback for node 69c500c7f63ce05331e2b2bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38200 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:49:25,897 - lava_callback - INFO - Received callback for node 69c50117f63ce05331e2b2c6 INFO: 172.20.0.1:44514 - "POST /node/69c50117f63ce05331e2b2c6 HTTP/1.0" 202 Accepted 2026-03-28 09:49:25,902 - lava_callback - INFO - Starting async job submit for node 69c50117f63ce05331e2b2c6 2026-03-28 09:49:25,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:49:25,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50117f63ce05331e2b2c6 HTTP/1.1" 200 None 2026-03-28 09:49:25,917 - lava_callback - DEBUG - Processing node 69c50117f63ce05331e2b2c6: baseline-arm64-clabbe 2026-03-28 09:49:25,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50117f63ce05331e2b2c6 2026-03-28 09:49:25,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:49:25,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50117f63ce05331e2b2c6 HTTP/1.1" 200 None 2026-03-28 09:49:25,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:49:25,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-28 09:49:25,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:49:25,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50117f63ce05331e2b2c6 HTTP/1.1" 200 None 2026-03-28 09:49:25,978 - lava_callback - ERROR - Error processing callback for node 69c50117f63ce05331e2b2c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43514 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:50:53,230 - lava_callback - INFO - Received callback for node 69c7a205be430e18c5d16828 INFO: 172.20.0.1:40176 - "POST /node/69c7a205be430e18c5d16828 HTTP/1.0" 202 Accepted 2026-03-28 09:50:53,237 - lava_callback - INFO - Starting async job submit for node 69c7a205be430e18c5d16828 2026-03-28 09:50:53,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:50:53,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a205be430e18c5d16828 HTTP/1.1" 200 None 2026-03-28 09:50:53,277 - lava_callback - DEBUG - Processing node 69c7a205be430e18c5d16828: baseline-arm64 2026-03-28 09:50:53,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:50:53,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:50:53,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a205be430e18c5d16828/log.txt.gz 2026-03-28 09:50:54,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:50:54,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:50:54,410 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a205be430e18c5d16828/lava-logs.yaml 2026-03-28 09:50:54,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:50:54,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:50:54,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a205be430e18c5d16828/lava_callback.json.gz 2026-03-28 09:50:54,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:50:54,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a205be430e18c5d16828 HTTP/1.1" 200 None 2026-03-28 09:50:54,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:50:54,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:50:54,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:50:54,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a205be430e18c5d16828 HTTP/1.1" 200 None 2026-03-28 09:50:54,916 - lava_callback - INFO - Completed processing callback for node 69c7a205be430e18c5d16828 INFO: 172.20.0.1:40186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35934 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:51:08,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:08,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46336 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:51:35,928 - lava_callback - INFO - Received callback for node 69c501a4f63ce05331e2b2d4 INFO: 172.20.0.1:46796 - "POST /node/69c501a4f63ce05331e2b2d4 HTTP/1.0" 202 Accepted 2026-03-28 09:51:35,933 - lava_callback - INFO - Starting async job submit for node 69c501a4f63ce05331e2b2d4 2026-03-28 09:51:35,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:35,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c501a4f63ce05331e2b2d4 HTTP/1.1" 200 None 2026-03-28 09:51:35,948 - lava_callback - DEBUG - Processing node 69c501a4f63ce05331e2b2d4: baseline-arm-baylibre 2026-03-28 09:51:35,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c501a4f63ce05331e2b2d4 2026-03-28 09:51:35,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:35,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c501a4f63ce05331e2b2d4 HTTP/1.1" 200 None 2026-03-28 09:51:35,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:35,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 09:51:35,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:36,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c501a4f63ce05331e2b2d4 HTTP/1.1" 200 None 2026-03-28 09:51:36,014 - lava_callback - ERROR - Error processing callback for node 69c501a4f63ce05331e2b2d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46806 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:51:40,365 - lava_callback - INFO - Received callback for node 69c7a474be430e18c5d1688c INFO: 172.20.0.1:36262 - "POST /node/69c7a474be430e18c5d1688c HTTP/1.0" 202 Accepted 2026-03-28 09:51:40,372 - lava_callback - INFO - Starting async job submit for node 69c7a474be430e18c5d1688c 2026-03-28 09:51:40,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:40,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a474be430e18c5d1688c HTTP/1.1" 200 None 2026-03-28 09:51:40,426 - lava_callback - DEBUG - Processing node 69c7a474be430e18c5d1688c: baseline-arm64 2026-03-28 09:51:40,558 - lava_callback - WARNING - Failed to upload log 2026-03-28 09:51:40,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:51:40,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:51:40,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a474be430e18c5d1688c/lava-logs.yaml 2026-03-28 09:51:40,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:51:40,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:51:40,941 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a474be430e18c5d1688c/lava_callback.json.gz 2026-03-28 09:51:40,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:40,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a474be430e18c5d1688c HTTP/1.1" 200 None 2026-03-28 09:51:40,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:41,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:51:41,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:51:41,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a474be430e18c5d1688c HTTP/1.1" 200 None 2026-03-28 09:51:41,110 - lava_callback - INFO - Completed processing callback for node 69c7a474be430e18c5d1688c INFO: 172.20.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40558 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:52:08,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:52:08,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53266 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 09:52:15,722 - lava_callback - INFO - Received callback for node 69c501ccf63ce05331e2b2e1 INFO: 172.20.0.1:53270 - "POST /node/69c501ccf63ce05331e2b2e1 HTTP/1.0" 202 Accepted 2026-03-28 09:52:15,728 - lava_callback - INFO - Starting async job submit for node 69c501ccf63ce05331e2b2e1 2026-03-28 09:52:15,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:52:15,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c501ccf63ce05331e2b2e1 HTTP/1.1" 200 None 2026-03-28 09:52:15,749 - lava_callback - DEBUG - Processing node 69c501ccf63ce05331e2b2e1: baseline-arm-baylibre 2026-03-28 09:52:15,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c501ccf63ce05331e2b2e1 2026-03-28 09:52:15,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:52:15,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c501ccf63ce05331e2b2e1 HTTP/1.1" 200 None 2026-03-28 09:52:15,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:52:15,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 09:52:15,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:52:15,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c501ccf63ce05331e2b2e1 HTTP/1.1" 200 None 2026-03-28 09:52:15,804 - lava_callback - ERROR - Error processing callback for node 69c501ccf63ce05331e2b2e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52138 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:54:09,522 - lava_callback - INFO - Received callback for node 69c7a0acbe430e18c5d167eb INFO: 172.20.0.1:52150 - "POST /node/69c7a0acbe430e18c5d167eb HTTP/1.0" 202 Accepted 2026-03-28 09:54:09,569 - lava_callback - INFO - Starting async job submit for node 69c7a0acbe430e18c5d167eb 2026-03-28 09:54:09,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:09,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a0acbe430e18c5d167eb HTTP/1.1" 200 None 2026-03-28 09:54:09,608 - lava_callback - DEBUG - Processing node 69c7a0acbe430e18c5d167eb: baseline-arm-pengutronix 2026-03-28 09:54:09,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:54:09,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:54:09,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7a0acbe430e18c5d167eb/log.txt.gz 2026-03-28 09:54:09,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:54:10,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:54:10,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7a0acbe430e18c5d167eb/lava-logs.yaml 2026-03-28 09:54:10,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:54:10,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:54:10,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7a0acbe430e18c5d167eb/lava_callback.json.gz DEBUG: Setup failed for 69c7a0acbe430e18c5d167eb. Transitting job node result: incomplete -> fail 2026-03-28 09:54:10,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:10,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a0acbe430e18c5d167eb HTTP/1.1" 200 None 2026-03-28 09:54:10,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:10,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 09:54:10,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:10,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a0acbe430e18c5d167eb HTTP/1.1" 200 None 2026-03-28 09:54:10,561 - lava_callback - INFO - Completed processing callback for node 69c7a0acbe430e18c5d167eb INFO: 172.20.0.1:52644 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 09:54:15,879 - lava_callback - INFO - Received callback for node 69c5023cf63ce05331e2b2f5 INFO: 172.20.0.1:52658 - "POST /node/69c5023cf63ce05331e2b2f5 HTTP/1.0" 202 Accepted 2026-03-28 09:54:15,885 - lava_callback - INFO - Starting async job submit for node 69c5023cf63ce05331e2b2f5 2026-03-28 09:54:15,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:15,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5023cf63ce05331e2b2f5 HTTP/1.1" 200 None 2026-03-28 09:54:15,900 - lava_callback - DEBUG - Processing node 69c5023cf63ce05331e2b2f5: baseline-arm-baylibre 2026-03-28 09:54:15,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5023cf63ce05331e2b2f5 2026-03-28 09:54:15,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:15,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5023cf63ce05331e2b2f5 HTTP/1.1" 200 None 2026-03-28 09:54:15,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:15,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 09:54:15,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:15,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5023cf63ce05331e2b2f5 HTTP/1.1" 200 None 2026-03-28 09:54:15,951 - lava_callback - ERROR - Error processing callback for node 69c5023cf63ce05331e2b2f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35208 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:54:28,533 - lava_callback - INFO - Received callback for node 69c7a512be430e18c5d168a8 INFO: 172.20.0.1:35216 - "POST /node/69c7a512be430e18c5d168a8 HTTP/1.0" 202 Accepted 2026-03-28 09:54:28,539 - lava_callback - INFO - Starting async job submit for node 69c7a512be430e18c5d168a8 2026-03-28 09:54:28,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:28,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a512be430e18c5d168a8 HTTP/1.1" 200 None 2026-03-28 09:54:28,594 - lava_callback - DEBUG - Processing node 69c7a512be430e18c5d168a8: baseline-arm64 2026-03-28 09:54:28,733 - lava_callback - WARNING - Failed to upload log 2026-03-28 09:54:28,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:54:29,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:54:29,022 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a512be430e18c5d168a8/lava-logs.yaml 2026-03-28 09:54:29,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 09:54:29,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 09:54:29,172 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a512be430e18c5d168a8/lava_callback.json.gz 2026-03-28 09:54:29,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:29,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a512be430e18c5d168a8 HTTP/1.1" 200 None 2026-03-28 09:54:29,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:29,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 09:54:29,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:29,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a512be430e18c5d168a8 HTTP/1.1" 200 None 2026-03-28 09:54:29,317 - lava_callback - INFO - Completed processing callback for node 69c7a512be430e18c5d168a8 INFO: 172.20.0.1:49200 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:54:38,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:54:38,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44844 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:55:06,497 - lava_callback - INFO - Received callback for node 69c50271f63ce05331e2b302 INFO: 172.20.0.1:52630 - "POST /node/69c50271f63ce05331e2b302 HTTP/1.0" 202 Accepted 2026-03-28 09:55:06,502 - lava_callback - INFO - Starting async job submit for node 69c50271f63ce05331e2b302 2026-03-28 09:55:06,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:55:06,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50271f63ce05331e2b302 HTTP/1.1" 200 None 2026-03-28 09:55:06,529 - lava_callback - DEBUG - Processing node 69c50271f63ce05331e2b302: baseline-arm64-clabbe 2026-03-28 09:55:06,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50271f63ce05331e2b302 2026-03-28 09:55:06,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:55:06,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50271f63ce05331e2b302 HTTP/1.1" 200 None 2026-03-28 09:55:06,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:55:06,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-28 09:55:06,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:55:06,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50271f63ce05331e2b302 HTTP/1.1" 200 None 2026-03-28 09:55:06,585 - lava_callback - ERROR - Error processing callback for node 69c50271f63ce05331e2b302: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50452 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:57:26,307 - lava_callback - INFO - Received callback for node 69c50300f63ce05331e2b30f INFO: 172.20.0.1:39294 - "POST /node/69c50300f63ce05331e2b30f HTTP/1.0" 202 Accepted 2026-03-28 09:57:26,313 - lava_callback - INFO - Starting async job submit for node 69c50300f63ce05331e2b30f 2026-03-28 09:57:26,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:57:26,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50300f63ce05331e2b30f HTTP/1.1" 200 None 2026-03-28 09:57:26,328 - lava_callback - DEBUG - Processing node 69c50300f63ce05331e2b30f: baseline-arm-clabbe 2026-03-28 09:57:26,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50300f63ce05331e2b30f 2026-03-28 09:57:26,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:57:26,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50300f63ce05331e2b30f HTTP/1.1" 200 None 2026-03-28 09:57:26,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:57:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 09:57:26,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:57:26,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50300f63ce05331e2b30f HTTP/1.1" 200 None 2026-03-28 09:57:26,387 - lava_callback - ERROR - Error processing callback for node 69c50300f63ce05331e2b30f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40376 - "GET / HTTP/1.0" 200 OK 2026-03-28 09:58:06,250 - lava_callback - INFO - Received callback for node 69c50325f63ce05331e2b31e INFO: 172.20.0.1:48844 - "POST /node/69c50325f63ce05331e2b31e HTTP/1.0" 202 Accepted 2026-03-28 09:58:06,255 - lava_callback - INFO - Starting async job submit for node 69c50325f63ce05331e2b31e 2026-03-28 09:58:06,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:58:06,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50325f63ce05331e2b31e HTTP/1.1" 200 None 2026-03-28 09:58:06,271 - lava_callback - DEBUG - Processing node 69c50325f63ce05331e2b31e: baseline-arm-clabbe 2026-03-28 09:58:06,271 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50325f63ce05331e2b31e 2026-03-28 09:58:06,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:58:06,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50325f63ce05331e2b31e HTTP/1.1" 200 None 2026-03-28 09:58:06,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:58:06,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 09:58:06,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 09:58:06,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50325f63ce05331e2b31e HTTP/1.1" 200 None 2026-03-28 09:58:06,364 - lava_callback - ERROR - Error processing callback for node 69c50325f63ce05331e2b31e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58152 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:00:06,590 - lava_callback - INFO - Received callback for node 69c50397f63ce05331e2b32e INFO: 172.20.0.1:58168 - "POST /node/69c50397f63ce05331e2b32e HTTP/1.0" 202 Accepted 2026-03-28 10:00:06,596 - lava_callback - INFO - Starting async job submit for node 69c50397f63ce05331e2b32e 2026-03-28 10:00:06,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:06,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50397f63ce05331e2b32e HTTP/1.1" 200 None 2026-03-28 10:00:06,613 - lava_callback - DEBUG - Processing node 69c50397f63ce05331e2b32e: baseline-arm-clabbe 2026-03-28 10:00:06,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50397f63ce05331e2b32e 2026-03-28 10:00:06,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:06,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50397f63ce05331e2b32e HTTP/1.1" 200 None 2026-03-28 10:00:06,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:06,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 10:00:06,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:06,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50397f63ce05331e2b32e HTTP/1.1" 200 None 2026-03-28 10:00:06,734 - lava_callback - ERROR - Error processing callback for node 69c50397f63ce05331e2b32e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47866 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:00:21,511 - lava_callback - INFO - Received callback for node 69c7a676be430e18c5d168ea INFO: 172.20.0.1:54358 - "POST /node/69c7a676be430e18c5d168ea HTTP/1.0" 202 Accepted 2026-03-28 10:00:21,517 - lava_callback - INFO - Starting async job submit for node 69c7a676be430e18c5d168ea 2026-03-28 10:00:21,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:21,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a676be430e18c5d168ea HTTP/1.1" 200 None 2026-03-28 10:00:21,571 - lava_callback - DEBUG - Processing node 69c7a676be430e18c5d168ea: baseline-arm64 2026-03-28 10:00:21,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:00:22,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:00:22,040 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a676be430e18c5d168ea/log.txt.gz 2026-03-28 10:00:22,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:00:22,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:00:22,449 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a676be430e18c5d168ea/lava-logs.yaml 2026-03-28 10:00:22,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:00:22,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:00:22,627 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7a676be430e18c5d168ea/lava_callback.json.gz 2026-03-28 10:00:22,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:22,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a676be430e18c5d168ea HTTP/1.1" 200 None 2026-03-28 10:00:22,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:22,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:00:22,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:22,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a676be430e18c5d168ea HTTP/1.1" 200 None 2026-03-28 10:00:22,709 - lava_callback - INFO - Completed processing callback for node 69c7a676be430e18c5d168ea INFO: 172.20.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51780 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:00:38,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:38,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 10:00:39,263 - lava_callback - INFO - Received callback for node 69c7a5d2be430e18c5d168cf INFO: 172.20.0.1:51786 - "POST /node/69c7a5d2be430e18c5d168cf HTTP/1.0" 202 Accepted 2026-03-28 10:00:39,269 - lava_callback - INFO - Starting async job submit for node 69c7a5d2be430e18c5d168cf 2026-03-28 10:00:39,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:39,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a5d2be430e18c5d168cf HTTP/1.1" 200 None 2026-03-28 10:00:39,314 - lava_callback - DEBUG - Processing node 69c7a5d2be430e18c5d168cf: ltp-watchqueue 2026-03-28 10:00:39,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:00:39,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:00:39,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c7a5d2be430e18c5d168cf/log.txt.gz 2026-03-28 10:00:40,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:00:40,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:00:40,203 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c7a5d2be430e18c5d168cf/lava-logs.yaml 2026-03-28 10:00:40,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:00:40,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:00:40,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c7a5d2be430e18c5d168cf/lava_callback.json.gz 2026-03-28 10:00:40,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:40,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a5d2be430e18c5d168cf HTTP/1.1" 200 None 2026-03-28 10:00:40,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:40,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 10:00:40,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:00:40,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a5d2be430e18c5d168cf HTTP/1.1" 200 None 2026-03-28 10:00:40,486 - lava_callback - INFO - Completed processing callback for node 69c7a5d2be430e18c5d168cf INFO: 172.20.0.1:52230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51430 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:01:01,718 - lava_callback - INFO - Received callback for node 69c7a32cbe430e18c5d16855 INFO: 172.20.0.1:59410 - "POST /node/69c7a32cbe430e18c5d16855 HTTP/1.0" 202 Accepted 2026-03-28 10:01:01,725 - lava_callback - INFO - Starting async job submit for node 69c7a32cbe430e18c5d16855 2026-03-28 10:01:01,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:01:01,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a32cbe430e18c5d16855 HTTP/1.1" 200 None 2026-03-28 10:01:01,783 - lava_callback - DEBUG - Processing node 69c7a32cbe430e18c5d16855: baseline-x86 2026-03-28 10:01:02,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:01:02,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:01:02,645 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c7a32cbe430e18c5d16855/log.txt.gz 2026-03-28 10:01:03,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:01:03,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:01:03,313 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c7a32cbe430e18c5d16855/lava-logs.yaml 2026-03-28 10:01:03,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:01:03,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:01:03,497 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c7a32cbe430e18c5d16855/lava_callback.json.gz DEBUG: Setup failed for 69c7a32cbe430e18c5d16855. Transitting job node result: incomplete -> fail 2026-03-28 10:01:03,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:01:03,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a32cbe430e18c5d16855 HTTP/1.1" 200 None 2026-03-28 10:01:03,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:01:03,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 10:01:03,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:01:03,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a32cbe430e18c5d16855 HTTP/1.1" 200 None 2026-03-28 10:01:03,735 - lava_callback - INFO - Completed processing callback for node 69c7a32cbe430e18c5d16855 INFO: 172.20.0.1:59424 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:01:08,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:01:08,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50684 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:02:06,402 - lava_callback - INFO - Received callback for node 69c50409f63ce05331e2b344 INFO: 172.20.0.1:60462 - "POST /node/69c50409f63ce05331e2b344 HTTP/1.0" 202 Accepted 2026-03-28 10:02:06,407 - lava_callback - INFO - Starting async job submit for node 69c50409f63ce05331e2b344 2026-03-28 10:02:06,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:06,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50409f63ce05331e2b344 HTTP/1.1" 200 None 2026-03-28 10:02:06,424 - lava_callback - DEBUG - Processing node 69c50409f63ce05331e2b344: baseline-arm-clabbe 2026-03-28 10:02:06,424 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50409f63ce05331e2b344 2026-03-28 10:02:06,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:06,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50409f63ce05331e2b344 HTTP/1.1" 200 None 2026-03-28 10:02:06,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:06,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 10:02:06,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:06,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50409f63ce05331e2b344 HTTP/1.1" 200 None 2026-03-28 10:02:06,483 - lava_callback - ERROR - Error processing callback for node 69c50409f63ce05331e2b344: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38282 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:02:26,347 - lava_callback - INFO - Received callback for node 69c5042ef63ce05331e2b350 INFO: 172.20.0.1:45244 - "POST /node/69c5042ef63ce05331e2b350 HTTP/1.0" 202 Accepted 2026-03-28 10:02:26,352 - lava_callback - INFO - Starting async job submit for node 69c5042ef63ce05331e2b350 2026-03-28 10:02:26,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:26,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5042ef63ce05331e2b350 HTTP/1.1" 200 None 2026-03-28 10:02:26,368 - lava_callback - DEBUG - Processing node 69c5042ef63ce05331e2b350: baseline-arm-clabbe 2026-03-28 10:02:26,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5042ef63ce05331e2b350 2026-03-28 10:02:26,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:26,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5042ef63ce05331e2b350 HTTP/1.1" 200 None 2026-03-28 10:02:26,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:26,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-28 10:02:26,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:02:26,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5042ef63ce05331e2b350 HTTP/1.1" 200 None 2026-03-28 10:02:26,423 - lava_callback - ERROR - Error processing callback for node 69c5042ef63ce05331e2b350: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:04:46,534 - lava_callback - INFO - Received callback for node 69c504b2f63ce05331e2b369 INFO: 172.20.0.1:36500 - "POST /node/69c504b2f63ce05331e2b369 HTTP/1.0" 202 Accepted 2026-03-28 10:04:46,540 - lava_callback - INFO - Starting async job submit for node 69c504b2f63ce05331e2b369 2026-03-28 10:04:46,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:04:46,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c504b2f63ce05331e2b369 HTTP/1.1" 200 None 2026-03-28 10:04:46,555 - lava_callback - DEBUG - Processing node 69c504b2f63ce05331e2b369: baseline-arm64-clabbe 2026-03-28 10:04:46,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c504b2f63ce05331e2b369 2026-03-28 10:04:46,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:04:46,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c504b2f63ce05331e2b369 HTTP/1.1" 200 None 2026-03-28 10:04:46,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:04:46,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-28 10:04:46,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:04:46,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c504b2f63ce05331e2b369 HTTP/1.1" 200 None 2026-03-28 10:04:46,606 - lava_callback - ERROR - Error processing callback for node 69c504b2f63ce05331e2b369: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41294 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:08:10,390 - lava_callback - INFO - Received callback for node 69c7a471be430e18c5d16889 INFO: 172.20.0.1:46244 - "POST /node/69c7a471be430e18c5d16889 HTTP/1.0" 202 Accepted 2026-03-28 10:08:10,396 - lava_callback - INFO - Starting async job submit for node 69c7a471be430e18c5d16889 2026-03-28 10:08:10,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:10,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a471be430e18c5d16889 HTTP/1.1" 200 None 2026-03-28 10:08:10,471 - lava_callback - DEBUG - Processing node 69c7a471be430e18c5d16889: ltp-smoke 2026-03-28 10:08:10,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:08:11,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:08:11,158 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7a471be430e18c5d16889/log.txt.gz 2026-03-28 10:08:11,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:08:11,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:08:11,678 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7a471be430e18c5d16889/lava-logs.yaml 2026-03-28 10:08:11,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:08:11,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:08:11,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7a471be430e18c5d16889/lava_callback.json.gz 2026-03-28 10:08:11,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:11,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a471be430e18c5d16889 HTTP/1.1" 200 None 2026-03-28 10:08:11,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:12,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb1be430e18c5d159e5 HTTP/1.1" 200 None 2026-03-28 10:08:12,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:12,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a471be430e18c5d16889 HTTP/1.1" 200 None 2026-03-28 10:08:12,172 - lava_callback - INFO - Completed processing callback for node 69c7a471be430e18c5d16889 INFO: 172.20.0.1:46258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38194 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:08:38,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:38,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37230 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:08:46,725 - lava_callback - INFO - Received callback for node 69c5059ff63ce05331e2b392 INFO: 172.20.0.1:37232 - "POST /node/69c5059ff63ce05331e2b392 HTTP/1.0" 202 Accepted 2026-03-28 10:08:46,730 - lava_callback - INFO - Starting async job submit for node 69c5059ff63ce05331e2b392 2026-03-28 10:08:46,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:46,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5059ff63ce05331e2b392 HTTP/1.1" 200 None 2026-03-28 10:08:46,747 - lava_callback - DEBUG - Processing node 69c5059ff63ce05331e2b392: baseline-arm64-clabbe 2026-03-28 10:08:46,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5059ff63ce05331e2b392 2026-03-28 10:08:46,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:46,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5059ff63ce05331e2b392 HTTP/1.1" 200 None 2026-03-28 10:08:46,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:46,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-28 10:08:46,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:08:46,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5059ff63ce05331e2b392 HTTP/1.1" 200 None 2026-03-28 10:08:46,802 - lava_callback - ERROR - Error processing callback for node 69c5059ff63ce05331e2b392: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:14:06,113 - lava_callback - INFO - Received callback for node 69c7a76cbe430e18c5d16921 INFO: 172.20.0.1:38262 - "POST /node/69c7a76cbe430e18c5d16921 HTTP/1.0" 202 Accepted 2026-03-28 10:14:06,120 - lava_callback - INFO - Starting async job submit for node 69c7a76cbe430e18c5d16921 2026-03-28 10:14:06,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:14:06,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a76cbe430e18c5d16921 HTTP/1.1" 200 None 2026-03-28 10:14:06,160 - lava_callback - DEBUG - Processing node 69c7a76cbe430e18c5d16921: baseline-nfs-arm64 INFO: 172.20.0.1:38270 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:14:06,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:14:06,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:14:06,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a76cbe430e18c5d16921/log.txt.gz 2026-03-28 10:14:07,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:14:07,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:14:07,350 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a76cbe430e18c5d16921/lava-logs.yaml 2026-03-28 10:14:07,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:14:07,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:14:07,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a76cbe430e18c5d16921/lava_callback.json.gz 2026-03-28 10:14:07,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:14:07,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a76cbe430e18c5d16921 HTTP/1.1" 200 None 2026-03-28 10:14:07,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:14:07,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:14:07,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:14:07,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a76cbe430e18c5d16921 HTTP/1.1" 200 None 2026-03-28 10:14:07,840 - lava_callback - INFO - Completed processing callback for node 69c7a76cbe430e18c5d16921 2026-03-28 10:14:08,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:14:08,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58410 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:16:32,853 - lava_callback - INFO - Received callback for node 69c7a5dbbe430e18c5d168d5 INFO: 172.20.0.1:45022 - "POST /node/69c7a5dbbe430e18c5d168d5 HTTP/1.0" 202 Accepted 2026-03-28 10:16:32,861 - lava_callback - INFO - Starting async job submit for node 69c7a5dbbe430e18c5d168d5 2026-03-28 10:16:32,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:16:32,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a5dbbe430e18c5d168d5 HTTP/1.1" 200 None 2026-03-28 10:16:32,922 - lava_callback - DEBUG - Processing node 69c7a5dbbe430e18c5d168d5: baseline-nfs-arm64 2026-03-28 10:16:33,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:16:34,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:16:34,297 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a5dbbe430e18c5d168d5/log.txt.gz 2026-03-28 10:16:34,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:16:35,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:16:35,183 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a5dbbe430e18c5d168d5/lava-logs.yaml 2026-03-28 10:16:35,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:16:35,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:16:35,541 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7a5dbbe430e18c5d168d5/lava_callback.json.gz DEBUG: Setup failed for 69c7a5dbbe430e18c5d168d5. Transitting job node result: incomplete -> fail 2026-03-28 10:16:35,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:16:35,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a5dbbe430e18c5d168d5 HTTP/1.1" 200 None 2026-03-28 10:16:35,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:16:35,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:16:35,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:16:35,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a5dbbe430e18c5d168d5 HTTP/1.1" 200 None 2026-03-28 10:16:35,778 - lava_callback - INFO - Completed processing callback for node 69c7a5dbbe430e18c5d168d5 INFO: 172.20.0.1:45030 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:16:38,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:16:38,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35868 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:21:26,689 - lava_callback - INFO - Received callback for node 69c7ab8cbe430e18c5d169b4 INFO: 172.20.0.1:35872 - "POST /node/69c7ab8cbe430e18c5d169b4 HTTP/1.0" 202 Accepted 2026-03-28 10:21:26,695 - lava_callback - INFO - Starting async job submit for node 69c7ab8cbe430e18c5d169b4 2026-03-28 10:21:26,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:21:26,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ab8cbe430e18c5d169b4 HTTP/1.1" 200 None 2026-03-28 10:21:26,714 - lava_callback - DEBUG - Processing node 69c7ab8cbe430e18c5d169b4: baseline-nfs-arm64 2026-03-28 10:21:26,715 - lava_callback - WARNING - Failed to upload log 2026-03-28 10:21:26,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:21:26,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:21:26,868 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7ab8cbe430e18c5d169b4/lava-logs.yaml 2026-03-28 10:21:26,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:21:27,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:21:27,172 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7ab8cbe430e18c5d169b4/lava_callback.json.gz 2026-03-28 10:21:27,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:21:27,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ab8cbe430e18c5d169b4 HTTP/1.1" 200 None 2026-03-28 10:21:27,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:21:27,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:21:27,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:21:27,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ab8cbe430e18c5d169b4 HTTP/1.1" 200 None 2026-03-28 10:21:27,236 - lava_callback - INFO - Completed processing callback for node 69c7ab8cbe430e18c5d169b4 INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:21:38,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:21:38,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:24:03,453 - lava_callback - INFO - Received callback for node 69c7a845be430e18c5d16937 INFO: 172.20.0.1:58742 - "POST /node/69c7a845be430e18c5d16937 HTTP/1.0" 202 Accepted 2026-03-28 10:24:03,464 - lava_callback - INFO - Starting async job submit for node 69c7a845be430e18c5d16937 2026-03-28 10:24:03,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:24:03,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a845be430e18c5d16937 HTTP/1.1" 200 None 2026-03-28 10:24:03,534 - lava_callback - DEBUG - Processing node 69c7a845be430e18c5d16937: baseline-arm 2026-03-28 10:24:04,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:24:04,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:24:04,324 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7a845be430e18c5d16937/log.txt.gz 2026-03-28 10:24:04,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:24:04,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:24:04,993 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7a845be430e18c5d16937/lava-logs.yaml 2026-03-28 10:24:05,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:24:05,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:24:05,187 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7a845be430e18c5d16937/lava_callback.json.gz DEBUG: Setup failed for 69c7a845be430e18c5d16937. Transitting job node result: incomplete -> fail 2026-03-28 10:24:05,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:24:05,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7a845be430e18c5d16937 HTTP/1.1" 200 None 2026-03-28 10:24:05,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:24:05,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 10:24:05,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:24:05,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7a845be430e18c5d16937 HTTP/1.1" 200 None 2026-03-28 10:24:05,565 - lava_callback - INFO - Completed processing callback for node 69c7a845be430e18c5d16937 INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:24:08,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:24:08,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49910 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:25:57,975 - lava_callback - INFO - Received callback for node 69c7aaebbe430e18c5d169a3 INFO: 172.20.0.1:49924 - "POST /node/69c7aaebbe430e18c5d169a3 HTTP/1.0" 202 Accepted 2026-03-28 10:25:57,982 - lava_callback - INFO - Starting async job submit for node 69c7aaebbe430e18c5d169a3 2026-03-28 10:25:58,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:25:58,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aaebbe430e18c5d169a3 HTTP/1.1" 200 None 2026-03-28 10:25:58,032 - lava_callback - DEBUG - Processing node 69c7aaebbe430e18c5d169a3: baseline-arm64 2026-03-28 10:25:58,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:25:58,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:25:58,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7aaebbe430e18c5d169a3/log.txt.gz 2026-03-28 10:25:58,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:25:59,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:25:59,056 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7aaebbe430e18c5d169a3/lava-logs.yaml 2026-03-28 10:25:59,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:25:59,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:25:59,220 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7aaebbe430e18c5d169a3/lava_callback.json.gz DEBUG: Setup failed for 69c7aaebbe430e18c5d169a3. Transitting job node result: incomplete -> fail 2026-03-28 10:25:59,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:25:59,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aaebbe430e18c5d169a3 HTTP/1.1" 200 None 2026-03-28 10:25:59,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:25:59,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:25:59,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:25:59,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7aaebbe430e18c5d169a3 HTTP/1.1" 200 None 2026-03-28 10:25:59,442 - lava_callback - INFO - Completed processing callback for node 69c7aaebbe430e18c5d169a3 INFO: 172.20.0.1:35692 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:26:08,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:26:08,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49692 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:26:30,215 - lava_callback - INFO - Received callback for node 69c7ac97be430e18c5d169e6 INFO: 172.20.0.1:33496 - "POST /node/69c7ac97be430e18c5d169e6 HTTP/1.0" 202 Accepted 2026-03-28 10:26:30,222 - lava_callback - INFO - Starting async job submit for node 69c7ac97be430e18c5d169e6 2026-03-28 10:26:30,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:26:30,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ac97be430e18c5d169e6 HTTP/1.1" 200 None 2026-03-28 10:26:30,277 - lava_callback - DEBUG - Processing node 69c7ac97be430e18c5d169e6: baseline-arm64 2026-03-28 10:26:30,409 - lava_callback - WARNING - Failed to upload log 2026-03-28 10:26:30,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:26:30,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:26:30,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ac97be430e18c5d169e6/lava-logs.yaml 2026-03-28 10:26:30,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:26:30,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:26:30,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ac97be430e18c5d169e6/lava_callback.json.gz 2026-03-28 10:26:30,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:26:30,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ac97be430e18c5d169e6 HTTP/1.1" 200 None 2026-03-28 10:26:30,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:26:30,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:26:30,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:26:30,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ac97be430e18c5d169e6 HTTP/1.1" 200 None 2026-03-28 10:26:30,995 - lava_callback - INFO - Completed processing callback for node 69c7ac97be430e18c5d169e6 INFO: 172.20.0.1:33508 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:26:38,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:26:38,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41996 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:29:29,693 - lava_callback - INFO - Received callback for node 69c7ad54be430e18c5d16a1f INFO: 172.20.0.1:42006 - "POST /node/69c7ad54be430e18c5d16a1f HTTP/1.0" 202 Accepted 2026-03-28 10:29:29,700 - lava_callback - INFO - Starting async job submit for node 69c7ad54be430e18c5d16a1f 2026-03-28 10:29:29,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:29:29,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ad54be430e18c5d16a1f HTTP/1.1" 200 None 2026-03-28 10:29:29,744 - lava_callback - DEBUG - Processing node 69c7ad54be430e18c5d16a1f: baseline-arm64 2026-03-28 10:29:29,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:29:30,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:29:30,027 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ad54be430e18c5d16a1f/log.txt.gz 2026-03-28 10:29:30,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:29:30,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:29:30,296 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ad54be430e18c5d16a1f/lava-logs.yaml 2026-03-28 10:29:30,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:29:30,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:29:30,437 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ad54be430e18c5d16a1f/lava_callback.json.gz 2026-03-28 10:29:30,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:29:30,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ad54be430e18c5d16a1f HTTP/1.1" 200 None 2026-03-28 10:29:30,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:29:30,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:29:30,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:29:30,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ad54be430e18c5d16a1f HTTP/1.1" 200 None 2026-03-28 10:29:30,615 - lava_callback - INFO - Completed processing callback for node 69c7ad54be430e18c5d16a1f INFO: 172.20.0.1:36680 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:29:38,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:29:38,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55640 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:33:54,334 - lava_callback - INFO - Received callback for node 69c7ac26be430e18c5d169c8 INFO: 172.20.0.1:48478 - "POST /node/69c7ac26be430e18c5d169c8 HTTP/1.0" 202 Accepted 2026-03-28 10:33:54,341 - lava_callback - INFO - Starting async job submit for node 69c7ac26be430e18c5d169c8 2026-03-28 10:33:54,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:33:54,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ac26be430e18c5d169c8 HTTP/1.1" 200 None 2026-03-28 10:33:54,380 - lava_callback - DEBUG - Processing node 69c7ac26be430e18c5d169c8: baseline-arm64 2026-03-28 10:33:54,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:33:55,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:33:55,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ac26be430e18c5d169c8/log.txt.gz 2026-03-28 10:33:55,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:33:55,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:33:55,550 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ac26be430e18c5d169c8/lava-logs.yaml 2026-03-28 10:33:55,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:33:55,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:33:55,728 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7ac26be430e18c5d169c8/lava_callback.json.gz 2026-03-28 10:33:55,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:33:55,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ac26be430e18c5d169c8 HTTP/1.1" 200 None 2026-03-28 10:33:55,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:33:55,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:33:55,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:33:55,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ac26be430e18c5d169c8 HTTP/1.1" 200 None 2026-03-28 10:33:55,868 - lava_callback - INFO - Completed processing callback for node 69c7ac26be430e18c5d169c8 INFO: 172.20.0.1:48482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59692 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:34:08,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:34:08,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46710 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:41:19,876 - lava_callback - INFO - Received callback for node 69c72d0529015d0f9599238c INFO: 172.20.0.1:59344 - "POST /node/69c72d0529015d0f9599238c HTTP/1.0" 202 Accepted 2026-03-28 10:41:19,883 - lava_callback - INFO - Starting async job submit for node 69c72d0529015d0f9599238c 2026-03-28 10:41:19,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:41:19,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0529015d0f9599238c HTTP/1.1" 200 None 2026-03-28 10:41:19,923 - lava_callback - DEBUG - Processing node 69c72d0529015d0f9599238c: baseline-arm64-broonie 2026-03-28 10:41:20,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:41:20,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:41:20,371 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0529015d0f9599238c/log.txt.gz 2026-03-28 10:41:20,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:41:20,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:41:20,799 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0529015d0f9599238c/lava-logs.yaml 2026-03-28 10:41:20,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:41:21,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:41:21,127 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0529015d0f9599238c/lava_callback.json.gz 2026-03-28 10:41:21,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:41:21,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0529015d0f9599238c HTTP/1.1" 200 None 2026-03-28 10:41:21,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:41:21,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 10:41:21,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:41:21,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0529015d0f9599238c HTTP/1.1" 200 None 2026-03-28 10:41:21,216 - lava_callback - INFO - Completed processing callback for node 69c72d0529015d0f9599238c INFO: 172.20.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33364 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:41:38,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:41:38,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:42:47,223 - lava_callback - INFO - Received callback for node 69c7af71be430e18c5d16a9c INFO: 172.20.0.1:44446 - "POST /node/69c7af71be430e18c5d16a9c HTTP/1.0" 202 Accepted 2026-03-28 10:42:47,230 - lava_callback - INFO - Starting async job submit for node 69c7af71be430e18c5d16a9c 2026-03-28 10:42:47,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:42:47,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7af71be430e18c5d16a9c HTTP/1.1" 200 None 2026-03-28 10:42:47,288 - lava_callback - DEBUG - Processing node 69c7af71be430e18c5d16a9c: baseline-arm64 2026-03-28 10:42:47,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:42:47,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:42:47,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7af71be430e18c5d16a9c/log.txt.gz 2026-03-28 10:42:48,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:42:48,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:42:48,271 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7af71be430e18c5d16a9c/lava-logs.yaml 2026-03-28 10:42:48,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:42:48,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:42:48,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7af71be430e18c5d16a9c/lava_callback.json.gz DEBUG: Setup failed for 69c7af71be430e18c5d16a9c. Transitting job node result: incomplete -> fail 2026-03-28 10:42:48,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:42:48,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7af71be430e18c5d16a9c HTTP/1.1" 200 None 2026-03-28 10:42:48,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:42:48,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:42:48,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:42:48,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7af71be430e18c5d16a9c HTTP/1.1" 200 None 2026-03-28 10:42:48,677 - lava_callback - INFO - Completed processing callback for node 69c7af71be430e18c5d16a9c INFO: 172.20.0.1:58302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57362 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:43:08,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:43:08,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34046 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:43:19,969 - lava_callback - INFO - Received callback for node 69c7859f29015d0f959934b4 INFO: 172.20.0.1:53534 - "POST /node/69c7859f29015d0f959934b4 HTTP/1.0" 202 Accepted 2026-03-28 10:43:19,975 - lava_callback - INFO - Starting async job submit for node 69c7859f29015d0f959934b4 2026-03-28 10:43:19,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:43:20,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7859f29015d0f959934b4 HTTP/1.1" 200 None 2026-03-28 10:43:20,032 - lava_callback - DEBUG - Processing node 69c7859f29015d0f959934b4: baseline-arm64-broonie 2026-03-28 10:43:20,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:43:20,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:43:20,446 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7859f29015d0f959934b4/log.txt.gz 2026-03-28 10:43:20,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:43:20,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:43:20,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7859f29015d0f959934b4/lava-logs.yaml 2026-03-28 10:43:20,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:43:20,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:43:20,967 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7859f29015d0f959934b4/lava_callback.json.gz 2026-03-28 10:43:20,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:43:21,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7859f29015d0f959934b4 HTTP/1.1" 200 None 2026-03-28 10:43:21,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:43:21,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 10:43:21,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:43:21,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7859f29015d0f959934b4 HTTP/1.1" 200 None 2026-03-28 10:43:21,051 - lava_callback - INFO - Completed processing callback for node 69c7859f29015d0f959934b4 INFO: 172.20.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42556 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:43:38,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:43:38,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59570 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:45:20,252 - lava_callback - INFO - Received callback for node 69c79747be430e18c5d1642a INFO: 172.20.0.1:54108 - "POST /node/69c79747be430e18c5d1642a HTTP/1.0" 202 Accepted 2026-03-28 10:45:20,258 - lava_callback - INFO - Starting async job submit for node 69c79747be430e18c5d1642a 2026-03-28 10:45:20,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:45:20,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79747be430e18c5d1642a HTTP/1.1" 200 None 2026-03-28 10:45:20,330 - lava_callback - DEBUG - Processing node 69c79747be430e18c5d1642a: baseline-arm64-broonie 2026-03-28 10:45:20,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:45:20,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:45:20,790 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79747be430e18c5d1642a/log.txt.gz 2026-03-28 10:45:20,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:45:21,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:45:21,190 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79747be430e18c5d1642a/lava-logs.yaml 2026-03-28 10:45:21,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:45:21,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:45:21,390 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c79747be430e18c5d1642a/lava_callback.json.gz 2026-03-28 10:45:21,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:45:21,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79747be430e18c5d1642a HTTP/1.1" 200 None 2026-03-28 10:45:21,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:45:21,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:45:21,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:45:21,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79747be430e18c5d1642a HTTP/1.1" 200 None 2026-03-28 10:45:21,470 - lava_callback - INFO - Completed processing callback for node 69c79747be430e18c5d1642a INFO: 172.20.0.1:54110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:45:38,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:45:38,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58578 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:46:52,239 - lava_callback - INFO - Received callback for node 69c7ad15be430e18c5d16a04 INFO: 172.20.0.1:37564 - "POST /node/69c7ad15be430e18c5d16a04 HTTP/1.0" 202 Accepted 2026-03-28 10:46:52,285 - lava_callback - INFO - Starting async job submit for node 69c7ad15be430e18c5d16a04 2026-03-28 10:46:52,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:46:52,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ad15be430e18c5d16a04 HTTP/1.1" 200 None 2026-03-28 10:46:52,325 - lava_callback - DEBUG - Processing node 69c7ad15be430e18c5d16a04: baseline-arm-pengutronix 2026-03-28 10:46:52,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:46:52,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:46:52,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7ad15be430e18c5d16a04/log.txt.gz 2026-03-28 10:46:52,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:46:52,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:46:52,907 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7ad15be430e18c5d16a04/lava-logs.yaml 2026-03-28 10:46:52,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:46:53,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:46:53,051 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7ad15be430e18c5d16a04/lava_callback.json.gz DEBUG: Setup failed for 69c7ad15be430e18c5d16a04. Transitting job node result: incomplete -> fail 2026-03-28 10:46:53,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:46:53,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ad15be430e18c5d16a04 HTTP/1.1" 200 None 2026-03-28 10:46:53,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:46:53,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 10:46:53,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:46:53,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ad15be430e18c5d16a04 HTTP/1.1" 200 None 2026-03-28 10:46:53,197 - lava_callback - INFO - Completed processing callback for node 69c7ad15be430e18c5d16a04 INFO: 172.20.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33640 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:47:08,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:47:08,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55062 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:48:35,298 - lava_callback - INFO - Received callback for node 69c7ae4ebe430e18c5d16a55 INFO: 172.20.0.1:54038 - "POST /node/69c7ae4ebe430e18c5d16a55 HTTP/1.0" 202 Accepted 2026-03-28 10:48:35,305 - lava_callback - INFO - Starting async job submit for node 69c7ae4ebe430e18c5d16a55 2026-03-28 10:48:35,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:48:35,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae4ebe430e18c5d16a55 HTTP/1.1" 200 None 2026-03-28 10:48:35,372 - lava_callback - DEBUG - Processing node 69c7ae4ebe430e18c5d16a55: baseline-x86 2026-03-28 10:48:36,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:48:36,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:48:36,301 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c7ae4ebe430e18c5d16a55/log.txt.gz INFO: 172.20.0.1:54052 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:48:36,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:48:36,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:48:36,987 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c7ae4ebe430e18c5d16a55/lava-logs.yaml 2026-03-28 10:48:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:48:37,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:48:37,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c7ae4ebe430e18c5d16a55/lava_callback.json.gz DEBUG: Setup failed for 69c7ae4ebe430e18c5d16a55. Transitting job node result: incomplete -> fail 2026-03-28 10:48:37,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:48:37,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae4ebe430e18c5d16a55 HTTP/1.1" 200 None 2026-03-28 10:48:37,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:48:37,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 10:48:37,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:48:37,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ae4ebe430e18c5d16a55 HTTP/1.1" 200 None 2026-03-28 10:48:37,467 - lava_callback - INFO - Completed processing callback for node 69c7ae4ebe430e18c5d16a55 2026-03-28 10:48:38,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:48:38,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45368 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:50:14,973 - lava_callback - INFO - Received callback for node 69c7aff8be430e18c5d16aa6 INFO: 172.20.0.1:38754 - "POST /node/69c7aff8be430e18c5d16aa6 HTTP/1.0" 202 Accepted 2026-03-28 10:50:14,980 - lava_callback - INFO - Starting async job submit for node 69c7aff8be430e18c5d16aa6 2026-03-28 10:50:15,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:50:15,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aff8be430e18c5d16aa6 HTTP/1.1" 200 None 2026-03-28 10:50:15,018 - lava_callback - DEBUG - Processing node 69c7aff8be430e18c5d16aa6: baseline-arm64 INFO: 172.20.0.1:38764 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 10:50:15,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:50:15,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:50:15,699 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7aff8be430e18c5d16aa6/log.txt.gz 2026-03-28 10:50:15,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:50:16,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:50:16,187 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7aff8be430e18c5d16aa6/lava-logs.yaml 2026-03-28 10:50:16,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:50:16,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:50:16,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7aff8be430e18c5d16aa6/lava_callback.json.gz 2026-03-28 10:50:16,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:50:16,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aff8be430e18c5d16aa6 HTTP/1.1" 200 None 2026-03-28 10:50:16,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:50:16,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:50:16,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:50:16,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7aff8be430e18c5d16aa6 HTTP/1.1" 200 None INFO: 172.20.0.1:38770 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:50:16,535 - lava_callback - INFO - Completed processing callback for node 69c7aff8be430e18c5d16aa6 INFO: 172.20.0.1:37994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51400 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:50:38,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:50:38,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:51:12,564 - lava_callback - INFO - Received callback for node 69c7ae9cbe430e18c5d16a7a INFO: 172.20.0.1:59968 - "POST /node/69c7ae9cbe430e18c5d16a7a HTTP/1.0" 202 Accepted 2026-03-28 10:51:12,570 - lava_callback - INFO - Starting async job submit for node 69c7ae9cbe430e18c5d16a7a 2026-03-28 10:51:12,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:51:12,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae9cbe430e18c5d16a7a HTTP/1.1" 200 None 2026-03-28 10:51:12,652 - lava_callback - DEBUG - Processing node 69c7ae9cbe430e18c5d16a7a: ltp-smoke 2026-03-28 10:51:13,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:51:13,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:51:13,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7ae9cbe430e18c5d16a7a/log.txt.gz 2026-03-28 10:51:13,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:51:13,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:51:13,790 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7ae9cbe430e18c5d16a7a/lava-logs.yaml 2026-03-28 10:51:13,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:51:14,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:51:14,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7ae9cbe430e18c5d16a7a/lava_callback.json.gz 2026-03-28 10:51:14,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:51:14,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ae9cbe430e18c5d16a7a HTTP/1.1" 200 None 2026-03-28 10:51:14,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:51:14,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb1be430e18c5d159e5 HTTP/1.1" 200 None 2026-03-28 10:51:14,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:51:14,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7ae9cbe430e18c5d16a7a HTTP/1.1" 200 None 2026-03-28 10:51:14,313 - lava_callback - INFO - Completed processing callback for node 69c7ae9cbe430e18c5d16a7a INFO: 172.20.0.1:59970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48830 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:51:38,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:51:38,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48496 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:52:19,211 - lava_callback - INFO - Received callback for node 69c50fe0f63ce05331e2b501 INFO: 172.20.0.1:48500 - "POST /node/69c50fe0f63ce05331e2b501 HTTP/1.0" 202 Accepted 2026-03-28 10:52:19,216 - lava_callback - INFO - Starting async job submit for node 69c50fe0f63ce05331e2b501 2026-03-28 10:52:19,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:52:19,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50fe0f63ce05331e2b501 HTTP/1.1" 200 None 2026-03-28 10:52:19,233 - lava_callback - DEBUG - Processing node 69c50fe0f63ce05331e2b501: baseline-arm-baylibre 2026-03-28 10:52:19,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c50fe0f63ce05331e2b501 2026-03-28 10:52:19,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:52:19,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c50fe0f63ce05331e2b501 HTTP/1.1" 200 None 2026-03-28 10:52:19,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:52:19,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 10:52:19,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:52:19,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c50fe0f63ce05331e2b501 HTTP/1.1" 200 None 2026-03-28 10:52:19,285 - lava_callback - ERROR - Error processing callback for node 69c50fe0f63ce05331e2b501: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58264 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:53:13,080 - lava_callback - INFO - Received callback for node 69c7af3bbe430e18c5d16a8e INFO: 172.20.0.1:57572 - "POST /node/69c7af3bbe430e18c5d16a8e HTTP/1.0" 202 Accepted 2026-03-28 10:53:13,087 - lava_callback - INFO - Starting async job submit for node 69c7af3bbe430e18c5d16a8e 2026-03-28 10:53:13,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:53:13,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7af3bbe430e18c5d16a8e HTTP/1.1" 200 None 2026-03-28 10:53:13,166 - lava_callback - DEBUG - Processing node 69c7af3bbe430e18c5d16a8e: baseline-arm 2026-03-28 10:53:13,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:53:13,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:53:13,901 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7af3bbe430e18c5d16a8e/log.txt.gz 2026-03-28 10:53:14,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:53:14,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:53:14,503 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7af3bbe430e18c5d16a8e/lava-logs.yaml 2026-03-28 10:53:14,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:53:14,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:53:14,693 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7af3bbe430e18c5d16a8e/lava_callback.json.gz DEBUG: Setup failed for 69c7af3bbe430e18c5d16a8e. Transitting job node result: incomplete -> fail 2026-03-28 10:53:14,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:53:14,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7af3bbe430e18c5d16a8e HTTP/1.1" 200 None 2026-03-28 10:53:14,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:53:14,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 10:53:14,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:53:14,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7af3bbe430e18c5d16a8e HTTP/1.1" 200 None 2026-03-28 10:53:14,982 - lava_callback - INFO - Completed processing callback for node 69c7af3bbe430e18c5d16a8e INFO: 172.20.0.1:57588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54118 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:53:38,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:53:38,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58996 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:55:18,536 - lava_callback - INFO - Received callback for node 69c7aef3be430e18c5d16a84 INFO: 172.20.0.1:59010 - "POST /node/69c7aef3be430e18c5d16a84 HTTP/1.0" 202 Accepted 2026-03-28 10:55:18,544 - lava_callback - INFO - Starting async job submit for node 69c7aef3be430e18c5d16a84 2026-03-28 10:55:18,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:55:18,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aef3be430e18c5d16a84 HTTP/1.1" 200 None 2026-03-28 10:55:18,606 - lava_callback - DEBUG - Processing node 69c7aef3be430e18c5d16a84: baseline-nfs-arm64 2026-03-28 10:55:19,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:55:19,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:55:19,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7aef3be430e18c5d16a84/log.txt.gz 2026-03-28 10:55:20,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:55:20,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:55:20,778 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7aef3be430e18c5d16a84/lava-logs.yaml 2026-03-28 10:55:20,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:55:20,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:55:20,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7aef3be430e18c5d16a84/lava_callback.json.gz DEBUG: Setup failed for 69c7aef3be430e18c5d16a84. Transitting job node result: incomplete -> fail 2026-03-28 10:55:21,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:55:21,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7aef3be430e18c5d16a84 HTTP/1.1" 200 None 2026-03-28 10:55:21,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:55:21,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 10:55:21,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:55:21,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7aef3be430e18c5d16a84 HTTP/1.1" 200 None 2026-03-28 10:55:21,260 - lava_callback - INFO - Completed processing callback for node 69c7aef3be430e18c5d16a84 INFO: 172.20.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44630 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:55:38,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:55:38,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:56:08,878 - lava_callback - INFO - Received callback for node 69c510c6f63ce05331e2b553 INFO: 172.20.0.1:39256 - "POST /node/69c510c6f63ce05331e2b553 HTTP/1.0" 202 Accepted 2026-03-28 10:56:08,884 - lava_callback - INFO - Starting async job submit for node 69c510c6f63ce05331e2b553 2026-03-28 10:56:08,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:08,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c510c6f63ce05331e2b553 HTTP/1.1" 200 None 2026-03-28 10:56:08,900 - lava_callback - DEBUG - Processing node 69c510c6f63ce05331e2b553: baseline-arm-clabbe 2026-03-28 10:56:08,901 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c510c6f63ce05331e2b553 2026-03-28 10:56:08,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:08,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c510c6f63ce05331e2b553 HTTP/1.1" 200 None 2026-03-28 10:56:08,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 10:56:08,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:08,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c510c6f63ce05331e2b553 HTTP/1.1" 200 None 2026-03-28 10:56:08,970 - lava_callback - ERROR - Error processing callback for node 69c510c6f63ce05331e2b553: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:56:48,769 - lava_callback - INFO - Received callback for node 69c510edf63ce05331e2b566 INFO: 172.20.0.1:49248 - "POST /node/69c510edf63ce05331e2b566 HTTP/1.0" 202 Accepted 2026-03-28 10:56:48,775 - lava_callback - INFO - Starting async job submit for node 69c510edf63ce05331e2b566 2026-03-28 10:56:48,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:48,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c510edf63ce05331e2b566 HTTP/1.1" 200 None 2026-03-28 10:56:48,792 - lava_callback - DEBUG - Processing node 69c510edf63ce05331e2b566: baseline-arm-clabbe 2026-03-28 10:56:48,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c510edf63ce05331e2b566 2026-03-28 10:56:48,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:48,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c510edf63ce05331e2b566 HTTP/1.1" 200 None 2026-03-28 10:56:48,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:48,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 10:56:48,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:56:48,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c510edf63ce05331e2b566 HTTP/1.1" 200 None 2026-03-28 10:56:48,887 - lava_callback - ERROR - Error processing callback for node 69c510edf63ce05331e2b566: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48954 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:57:28,836 - lava_callback - INFO - Received callback for node 69c51114f63ce05331e2b576 INFO: 172.20.0.1:48966 - "POST /node/69c51114f63ce05331e2b576 HTTP/1.0" 202 Accepted 2026-03-28 10:57:28,842 - lava_callback - INFO - Starting async job submit for node 69c51114f63ce05331e2b576 2026-03-28 10:57:28,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:28,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51114f63ce05331e2b576 HTTP/1.1" 200 None 2026-03-28 10:57:28,859 - lava_callback - DEBUG - Processing node 69c51114f63ce05331e2b576: baseline-arm-clabbe 2026-03-28 10:57:28,859 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c51114f63ce05331e2b576 2026-03-28 10:57:28,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:28,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51114f63ce05331e2b576 HTTP/1.1" 200 None 2026-03-28 10:57:28,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:28,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 10:57:28,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:28,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51114f63ce05331e2b576 HTTP/1.1" 200 None 2026-03-28 10:57:28,921 - lava_callback - ERROR - Error processing callback for node 69c51114f63ce05331e2b576: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55986 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:57:54,495 - lava_callback - INFO - Received callback for node 69c7b242be430e18c5d16ae9 INFO: 172.20.0.1:44168 - "POST /node/69c7b242be430e18c5d16ae9 HTTP/1.0" 202 Accepted 2026-03-28 10:57:54,502 - lava_callback - INFO - Starting async job submit for node 69c7b242be430e18c5d16ae9 2026-03-28 10:57:54,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:54,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b242be430e18c5d16ae9 HTTP/1.1" 200 None 2026-03-28 10:57:54,547 - lava_callback - DEBUG - Processing node 69c7b242be430e18c5d16ae9: baseline-x86 2026-03-28 10:57:55,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:57:55,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:57:55,339 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c7b242be430e18c5d16ae9/log.txt.gz 2026-03-28 10:57:55,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:57:55,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:57:55,855 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c7b242be430e18c5d16ae9/lava-logs.yaml 2026-03-28 10:57:55,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 10:57:56,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 10:57:56,039 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c7b242be430e18c5d16ae9/lava_callback.json.gz 2026-03-28 10:57:56,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:56,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b242be430e18c5d16ae9 HTTP/1.1" 200 None 2026-03-28 10:57:56,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:56,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 10:57:56,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:57:56,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7b242be430e18c5d16ae9 HTTP/1.1" 200 None 2026-03-28 10:57:56,157 - lava_callback - INFO - Completed processing callback for node 69c7b242be430e18c5d16ae9 INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57840 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:58:08,855 - lava_callback - INFO - Received callback for node 69c5113af63ce05331e2b587 INFO: 172.20.0.1:57844 - "POST /node/69c5113af63ce05331e2b587 HTTP/1.0" 202 Accepted 2026-03-28 10:58:08,860 - lava_callback - INFO - Starting async job submit for node 69c5113af63ce05331e2b587 2026-03-28 10:58:08,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:08,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5113af63ce05331e2b587 HTTP/1.1" 200 None 2026-03-28 10:58:08,882 - lava_callback - DEBUG - Processing node 69c5113af63ce05331e2b587: baseline-arm-clabbe 2026-03-28 10:58:08,882 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5113af63ce05331e2b587 2026-03-28 10:58:08,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:08,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5113af63ce05331e2b587 HTTP/1.1" 200 None 2026-03-28 10:58:08,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:08,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 10:58:08,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:08,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:08,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5113af63ce05331e2b587 HTTP/1.1" 200 None 2026-03-28 10:58:08,937 - lava_callback - ERROR - Error processing callback for node 69c5113af63ce05331e2b587: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 10:58:08,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK 2026-03-28 10:58:48,976 - lava_callback - INFO - Received callback for node 69c51161f63ce05331e2b592 INFO: 172.20.0.1:48406 - "POST /node/69c51161f63ce05331e2b592 HTTP/1.0" 202 Accepted 2026-03-28 10:58:48,981 - lava_callback - INFO - Starting async job submit for node 69c51161f63ce05331e2b592 2026-03-28 10:58:48,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:48,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51161f63ce05331e2b592 HTTP/1.1" 200 None 2026-03-28 10:58:48,997 - lava_callback - DEBUG - Processing node 69c51161f63ce05331e2b592: baseline-arm-clabbe 2026-03-28 10:58:48,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c51161f63ce05331e2b592 2026-03-28 10:58:48,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:49,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51161f63ce05331e2b592 HTTP/1.1" 200 None 2026-03-28 10:58:49,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:49,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 10:58:49,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 10:58:49,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51161f63ce05331e2b592 HTTP/1.1" 200 None 2026-03-28 10:58:49,080 - lava_callback - ERROR - Error processing callback for node 69c51161f63ce05331e2b592: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36886 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:06:32,189 - lava_callback - INFO - Received callback for node 69c7b1aabe430e18c5d16acd INFO: 172.20.0.1:53804 - "POST /node/69c7b1aabe430e18c5d16acd HTTP/1.0" 202 Accepted 2026-03-28 11:06:32,237 - lava_callback - INFO - Starting async job submit for node 69c7b1aabe430e18c5d16acd 2026-03-28 11:06:32,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:06:32,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b1aabe430e18c5d16acd HTTP/1.1" 200 None 2026-03-28 11:06:32,276 - lava_callback - DEBUG - Processing node 69c7b1aabe430e18c5d16acd: baseline-arm-pengutronix 2026-03-28 11:06:32,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 11:06:32,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 11:06:32,549 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7b1aabe430e18c5d16acd/log.txt.gz 2026-03-28 11:06:32,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 11:06:32,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 11:06:32,818 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7b1aabe430e18c5d16acd/lava-logs.yaml 2026-03-28 11:06:32,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 11:06:32,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 11:06:32,961 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7b1aabe430e18c5d16acd/lava_callback.json.gz DEBUG: Setup failed for 69c7b1aabe430e18c5d16acd. Transitting job node result: incomplete -> fail 2026-03-28 11:06:33,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:06:33,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b1aabe430e18c5d16acd HTTP/1.1" 200 None 2026-03-28 11:06:33,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:06:33,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-03-28 11:06:33,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:06:33,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7b1aabe430e18c5d16acd HTTP/1.1" 200 None 2026-03-28 11:06:33,111 - lava_callback - INFO - Completed processing callback for node 69c7b1aabe430e18c5d16acd INFO: 172.20.0.1:53810 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:06:38,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:06:38,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46296 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:13:00,296 - lava_callback - INFO - Received callback for node 69c514b1f63ce05331e2b615 INFO: 172.20.0.1:35812 - "POST /node/69c514b1f63ce05331e2b615 HTTP/1.0" 202 Accepted 2026-03-28 11:13:00,301 - lava_callback - INFO - Starting async job submit for node 69c514b1f63ce05331e2b615 2026-03-28 11:13:00,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:13:00,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c514b1f63ce05331e2b615 HTTP/1.1" 200 None 2026-03-28 11:13:00,318 - lava_callback - DEBUG - Processing node 69c514b1f63ce05331e2b615: baseline-arm64-baylibre 2026-03-28 11:13:00,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c514b1f63ce05331e2b615 2026-03-28 11:13:00,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:13:00,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c514b1f63ce05331e2b615 HTTP/1.1" 200 None 2026-03-28 11:13:00,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:13:00,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:13:00,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:13:00,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c514b1f63ce05331e2b615 HTTP/1.1" 200 None 2026-03-28 11:13:00,383 - lava_callback - ERROR - Error processing callback for node 69c514b1f63ce05331e2b615: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:14:20,371 - lava_callback - INFO - Received callback for node 69c51500f63ce05331e2b622 INFO: 172.20.0.1:35224 - "POST /node/69c51500f63ce05331e2b622 HTTP/1.0" 202 Accepted 2026-03-28 11:14:20,377 - lava_callback - INFO - Starting async job submit for node 69c51500f63ce05331e2b622 2026-03-28 11:14:20,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:14:20,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51500f63ce05331e2b622 HTTP/1.1" 200 None 2026-03-28 11:14:20,392 - lava_callback - DEBUG - Processing node 69c51500f63ce05331e2b622: baseline-arm64-baylibre 2026-03-28 11:14:20,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c51500f63ce05331e2b622 2026-03-28 11:14:20,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:14:20,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51500f63ce05331e2b622 HTTP/1.1" 200 None 2026-03-28 11:14:20,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:14:20,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:14:20,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:14:20,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51500f63ce05331e2b622 HTTP/1.1" 200 None 2026-03-28 11:14:20,451 - lava_callback - ERROR - Error processing callback for node 69c51500f63ce05331e2b622: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:15:19,388 - lava_callback - INFO - Received callback for node 69c7b3a1be430e18c5d16b15 INFO: 172.20.0.1:53928 - "POST /node/69c7b3a1be430e18c5d16b15 HTTP/1.0" 202 Accepted 2026-03-28 11:15:19,396 - lava_callback - INFO - Starting async job submit for node 69c7b3a1be430e18c5d16b15 2026-03-28 11:15:19,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:15:19,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b3a1be430e18c5d16b15 HTTP/1.1" 200 None 2026-03-28 11:15:19,463 - lava_callback - DEBUG - Processing node 69c7b3a1be430e18c5d16b15: baseline-nfs-arm64 2026-03-28 11:15:20,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 11:15:20,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 11:15:20,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7b3a1be430e18c5d16b15/log.txt.gz 2026-03-28 11:15:21,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 11:15:21,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 11:15:21,545 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7b3a1be430e18c5d16b15/lava-logs.yaml 2026-03-28 11:15:21,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 11:15:21,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 11:15:21,775 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7b3a1be430e18c5d16b15/lava_callback.json.gz DEBUG: Setup failed for 69c7b3a1be430e18c5d16b15. Transitting job node result: incomplete -> fail 2026-03-28 11:15:21,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:15:21,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7b3a1be430e18c5d16b15 HTTP/1.1" 200 None 2026-03-28 11:15:21,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:15:21,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-03-28 11:15:21,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:15:21,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7b3a1be430e18c5d16b15 HTTP/1.1" 200 None 2026-03-28 11:15:22,030 - lava_callback - INFO - Completed processing callback for node 69c7b3a1be430e18c5d16b15 INFO: 172.20.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57470 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:15:39,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:15:39,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55366 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:17:00,494 - lava_callback - INFO - Received callback for node 69c5159ff63ce05331e2b639 INFO: 172.20.0.1:54440 - "POST /node/69c5159ff63ce05331e2b639 HTTP/1.0" 202 Accepted 2026-03-28 11:17:00,499 - lava_callback - INFO - Starting async job submit for node 69c5159ff63ce05331e2b639 2026-03-28 11:17:00,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:17:00,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5159ff63ce05331e2b639 HTTP/1.1" 200 None 2026-03-28 11:17:00,515 - lava_callback - DEBUG - Processing node 69c5159ff63ce05331e2b639: baseline-arm64-baylibre 2026-03-28 11:17:00,515 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5159ff63ce05331e2b639 2026-03-28 11:17:00,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:17:00,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5159ff63ce05331e2b639 HTTP/1.1" 200 None 2026-03-28 11:17:00,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:17:00,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:17:00,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:17:00,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5159ff63ce05331e2b639 HTTP/1.1" 200 None 2026-03-28 11:17:00,578 - lava_callback - ERROR - Error processing callback for node 69c5159ff63ce05331e2b639: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42750 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:18:20,624 - lava_callback - INFO - Received callback for node 69c515eff63ce05331e2b648 INFO: 172.20.0.1:35678 - "POST /node/69c515eff63ce05331e2b648 HTTP/1.0" 202 Accepted 2026-03-28 11:18:20,630 - lava_callback - INFO - Starting async job submit for node 69c515eff63ce05331e2b648 2026-03-28 11:18:20,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:18:20,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c515eff63ce05331e2b648 HTTP/1.1" 200 None 2026-03-28 11:18:20,646 - lava_callback - DEBUG - Processing node 69c515eff63ce05331e2b648: baseline-arm64-baylibre 2026-03-28 11:18:20,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c515eff63ce05331e2b648 2026-03-28 11:18:20,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:18:20,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c515eff63ce05331e2b648 HTTP/1.1" 200 None 2026-03-28 11:18:20,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:18:20,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:18:20,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:18:20,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c515eff63ce05331e2b648 HTTP/1.1" 200 None 2026-03-28 11:18:20,698 - lava_callback - ERROR - Error processing callback for node 69c515eff63ce05331e2b648: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42658 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:23:10,005 - lava_callback - INFO - Received callback for node 69c51712f63ce05331e2b671 INFO: 172.20.0.1:49040 - "POST /node/69c51712f63ce05331e2b671 HTTP/1.0" 202 Accepted 2026-03-28 11:23:10,011 - lava_callback - INFO - Starting async job submit for node 69c51712f63ce05331e2b671 2026-03-28 11:23:10,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:23:10,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51712f63ce05331e2b671 HTTP/1.1" 200 None 2026-03-28 11:23:10,027 - lava_callback - DEBUG - Processing node 69c51712f63ce05331e2b671: baseline-arm64-clabbe 2026-03-28 11:23:10,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c51712f63ce05331e2b671 2026-03-28 11:23:10,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:23:10,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51712f63ce05331e2b671 HTTP/1.1" 200 None 2026-03-28 11:23:10,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:23:10,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:23:10,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:23:10,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51712f63ce05331e2b671 HTTP/1.1" 200 None 2026-03-28 11:23:10,114 - lava_callback - ERROR - Error processing callback for node 69c51712f63ce05331e2b671: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:25:50,124 - lava_callback - INFO - Received callback for node 69c517aef63ce05331e2b685 INFO: 172.20.0.1:37178 - "POST /node/69c517aef63ce05331e2b685 HTTP/1.0" 202 Accepted 2026-03-28 11:25:50,129 - lava_callback - INFO - Starting async job submit for node 69c517aef63ce05331e2b685 2026-03-28 11:25:50,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:25:50,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c517aef63ce05331e2b685 HTTP/1.1" 200 None 2026-03-28 11:25:50,145 - lava_callback - DEBUG - Processing node 69c517aef63ce05331e2b685: baseline-arm64-clabbe 2026-03-28 11:25:50,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c517aef63ce05331e2b685 2026-03-28 11:25:50,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:25:50,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c517aef63ce05331e2b685 HTTP/1.1" 200 None 2026-03-28 11:25:50,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:25:50,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:25:50,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:25:50,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c517aef63ce05331e2b685 HTTP/1.1" 200 None 2026-03-28 11:25:50,206 - lava_callback - ERROR - Error processing callback for node 69c517aef63ce05331e2b685: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37078 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:27:10,206 - lava_callback - INFO - Received callback for node 69c517fcf63ce05331e2b68e INFO: 172.20.0.1:35266 - "POST /node/69c517fcf63ce05331e2b68e HTTP/1.0" 202 Accepted 2026-03-28 11:27:10,212 - lava_callback - INFO - Starting async job submit for node 69c517fcf63ce05331e2b68e 2026-03-28 11:27:10,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:27:10,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c517fcf63ce05331e2b68e HTTP/1.1" 200 None 2026-03-28 11:27:10,231 - lava_callback - DEBUG - Processing node 69c517fcf63ce05331e2b68e: baseline-arm64-clabbe 2026-03-28 11:27:10,232 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c517fcf63ce05331e2b68e 2026-03-28 11:27:10,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:27:10,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c517fcf63ce05331e2b68e HTTP/1.1" 200 None 2026-03-28 11:27:10,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:27:10,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:27:10,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:27:10,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c517fcf63ce05331e2b68e HTTP/1.1" 200 None 2026-03-28 11:27:10,283 - lava_callback - ERROR - Error processing callback for node 69c517fcf63ce05331e2b68e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33302 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:28:30,254 - lava_callback - INFO - Received callback for node 69c51849f63ce05331e2b6a3 INFO: 172.20.0.1:58378 - "POST /node/69c51849f63ce05331e2b6a3 HTTP/1.0" 202 Accepted 2026-03-28 11:28:30,260 - lava_callback - INFO - Starting async job submit for node 69c51849f63ce05331e2b6a3 2026-03-28 11:28:30,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:28:30,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51849f63ce05331e2b6a3 HTTP/1.1" 200 None 2026-03-28 11:28:30,276 - lava_callback - DEBUG - Processing node 69c51849f63ce05331e2b6a3: baseline-arm64-clabbe 2026-03-28 11:28:30,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c51849f63ce05331e2b6a3 2026-03-28 11:28:30,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:28:30,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51849f63ce05331e2b6a3 HTTP/1.1" 200 None 2026-03-28 11:28:30,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:28:30,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:28:30,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:28:30,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51849f63ce05331e2b6a3 HTTP/1.1" 200 None 2026-03-28 11:28:30,334 - lava_callback - ERROR - Error processing callback for node 69c51849f63ce05331e2b6a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45404 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:29:50,283 - lava_callback - INFO - Received callback for node 69c51898f63ce05331e2b6b1 INFO: 172.20.0.1:56666 - "POST /node/69c51898f63ce05331e2b6b1 HTTP/1.0" 202 Accepted 2026-03-28 11:29:50,289 - lava_callback - INFO - Starting async job submit for node 69c51898f63ce05331e2b6b1 2026-03-28 11:29:50,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:29:50,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51898f63ce05331e2b6b1 HTTP/1.1" 200 None 2026-03-28 11:29:50,308 - lava_callback - DEBUG - Processing node 69c51898f63ce05331e2b6b1: baseline-arm64-clabbe 2026-03-28 11:29:50,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c51898f63ce05331e2b6b1 2026-03-28 11:29:50,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:29:50,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c51898f63ce05331e2b6b1 HTTP/1.1" 200 None 2026-03-28 11:29:50,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:29:50,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:29:50,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:29:50,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c51898f63ce05331e2b6b1 HTTP/1.1" 200 None 2026-03-28 11:29:50,371 - lava_callback - ERROR - Error processing callback for node 69c51898f63ce05331e2b6b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43902 - "GET / HTTP/1.0" 200 OK 2026-03-28 11:30:50,305 - lava_callback - INFO - Received callback for node 69c518e6f63ce05331e2b6bb INFO: 172.20.0.1:47798 - "POST /node/69c518e6f63ce05331e2b6bb HTTP/1.0" 202 Accepted 2026-03-28 11:30:50,311 - lava_callback - INFO - Starting async job submit for node 69c518e6f63ce05331e2b6bb 2026-03-28 11:30:50,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:30:50,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c518e6f63ce05331e2b6bb HTTP/1.1" 200 None 2026-03-28 11:30:50,327 - lava_callback - DEBUG - Processing node 69c518e6f63ce05331e2b6bb: baseline-arm64-clabbe 2026-03-28 11:30:50,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c518e6f63ce05331e2b6bb 2026-03-28 11:30:50,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:30:50,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c518e6f63ce05331e2b6bb HTTP/1.1" 200 None 2026-03-28 11:30:50,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:30:50,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 11:30:50,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 11:30:50,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c518e6f63ce05331e2b6bb HTTP/1.1" 200 None 2026-03-28 11:30:50,575 - lava_callback - ERROR - Error processing callback for node 69c518e6f63ce05331e2b6bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60476 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:42:05,224 - lava_callback - INFO - Received callback for node 69c5298bf63ce05331e2b71a INFO: 172.20.0.1:37548 - "POST /node/69c5298bf63ce05331e2b71a HTTP/1.0" 202 Accepted 2026-03-28 12:42:05,229 - lava_callback - INFO - Starting async job submit for node 69c5298bf63ce05331e2b71a 2026-03-28 12:42:05,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:05,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5298bf63ce05331e2b71a HTTP/1.1" 200 None 2026-03-28 12:42:05,246 - lava_callback - DEBUG - Processing node 69c5298bf63ce05331e2b71a: baseline-arm-baylibre 2026-03-28 12:42:05,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5298bf63ce05331e2b71a 2026-03-28 12:42:05,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:05,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5298bf63ce05331e2b71a HTTP/1.1" 200 None 2026-03-28 12:42:05,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:05,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:42:05,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:05,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5298bf63ce05331e2b71a HTTP/1.1" 200 None 2026-03-28 12:42:05,307 - lava_callback - ERROR - Error processing callback for node 69c5298bf63ce05331e2b71a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:42:45,088 - lava_callback - INFO - Received callback for node 69c529b1f63ce05331e2b727 INFO: 172.20.0.1:59934 - "POST /node/69c529b1f63ce05331e2b727 HTTP/1.0" 202 Accepted 2026-03-28 12:42:45,093 - lava_callback - INFO - Starting async job submit for node 69c529b1f63ce05331e2b727 2026-03-28 12:42:45,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:45,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c529b1f63ce05331e2b727 HTTP/1.1" 200 None 2026-03-28 12:42:45,110 - lava_callback - DEBUG - Processing node 69c529b1f63ce05331e2b727: baseline-arm-baylibre 2026-03-28 12:42:45,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c529b1f63ce05331e2b727 2026-03-28 12:42:45,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:45,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c529b1f63ce05331e2b727 HTTP/1.1" 200 None 2026-03-28 12:42:45,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:45,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:42:45,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:42:45,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c529b1f63ce05331e2b727 HTTP/1.1" 200 None 2026-03-28 12:42:45,165 - lava_callback - ERROR - Error processing callback for node 69c529b1f63ce05331e2b727: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52808 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:45:13,584 - lava_callback - INFO - Received callback for node 69c52a49f63ce05331e2b747 INFO: 172.20.0.1:34790 - "POST /node/69c52a49f63ce05331e2b747 HTTP/1.0" 202 Accepted 2026-03-28 12:45:13,589 - lava_callback - INFO - Starting async job submit for node 69c52a49f63ce05331e2b747 2026-03-28 12:45:13,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:13,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52a49f63ce05331e2b747 HTTP/1.1" 200 None 2026-03-28 12:45:13,608 - lava_callback - DEBUG - Processing node 69c52a49f63ce05331e2b747: baseline-arm-clabbe 2026-03-28 12:45:13,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52a49f63ce05331e2b747 2026-03-28 12:45:13,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:13,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52a49f63ce05331e2b747 HTTP/1.1" 200 None 2026-03-28 12:45:13,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:13,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:45:13,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:13,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52a49f63ce05331e2b747 HTTP/1.1" 200 None 2026-03-28 12:45:13,667 - lava_callback - ERROR - Error processing callback for node 69c52a49f63ce05331e2b747: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51320 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:45:53,609 - lava_callback - INFO - Received callback for node 69c52a6ff63ce05331e2b750 INFO: 172.20.0.1:47470 - "POST /node/69c52a6ff63ce05331e2b750 HTTP/1.0" 202 Accepted 2026-03-28 12:45:53,615 - lava_callback - INFO - Starting async job submit for node 69c52a6ff63ce05331e2b750 2026-03-28 12:45:53,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:53,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52a6ff63ce05331e2b750 HTTP/1.1" 200 None 2026-03-28 12:45:53,632 - lava_callback - DEBUG - Processing node 69c52a6ff63ce05331e2b750: baseline-arm-clabbe 2026-03-28 12:45:53,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52a6ff63ce05331e2b750 2026-03-28 12:45:53,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:53,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52a6ff63ce05331e2b750 HTTP/1.1" 200 None 2026-03-28 12:45:53,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:53,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:45:53,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:45:53,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52a6ff63ce05331e2b750 HTTP/1.1" 200 None 2026-03-28 12:45:53,711 - lava_callback - ERROR - Error processing callback for node 69c52a6ff63ce05331e2b750: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56886 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:46:33,625 - lava_callback - INFO - Received callback for node 69c52a99f63ce05331e2b766 INFO: 172.20.0.1:34888 - "POST /node/69c52a99f63ce05331e2b766 HTTP/1.0" 202 Accepted 2026-03-28 12:46:33,630 - lava_callback - INFO - Starting async job submit for node 69c52a99f63ce05331e2b766 2026-03-28 12:46:33,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:46:33,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52a99f63ce05331e2b766 HTTP/1.1" 200 None 2026-03-28 12:46:33,646 - lava_callback - DEBUG - Processing node 69c52a99f63ce05331e2b766: baseline-arm-clabbe 2026-03-28 12:46:33,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52a99f63ce05331e2b766 2026-03-28 12:46:33,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:46:33,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52a99f63ce05331e2b766 HTTP/1.1" 200 None 2026-03-28 12:46:33,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:46:33,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:46:33,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:46:33,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52a99f63ce05331e2b766 HTTP/1.1" 200 None 2026-03-28 12:46:33,730 - lava_callback - ERROR - Error processing callback for node 69c52a99f63ce05331e2b766: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52936 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:48:13,702 - lava_callback - INFO - Received callback for node 69c52b08f63ce05331e2b798 INFO: 172.20.0.1:33820 - "POST /node/69c52b08f63ce05331e2b798 HTTP/1.0" 202 Accepted 2026-03-28 12:48:13,708 - lava_callback - INFO - Starting async job submit for node 69c52b08f63ce05331e2b798 2026-03-28 12:48:13,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:13,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52b08f63ce05331e2b798 HTTP/1.1" 200 None 2026-03-28 12:48:13,747 - lava_callback - DEBUG - Processing node 69c52b08f63ce05331e2b798: baseline-arm-clabbe 2026-03-28 12:48:13,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52b08f63ce05331e2b798 2026-03-28 12:48:13,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:13,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52b08f63ce05331e2b798 HTTP/1.1" 200 None 2026-03-28 12:48:13,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:13,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:48:13,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:13,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52b08f63ce05331e2b798 HTTP/1.1" 200 None 2026-03-28 12:48:13,809 - lava_callback - ERROR - Error processing callback for node 69c52b08f63ce05331e2b798: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60112 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:48:53,703 - lava_callback - INFO - Received callback for node 69c52b2ef63ce05331e2b7a7 INFO: 172.20.0.1:50660 - "POST /node/69c52b2ef63ce05331e2b7a7 HTTP/1.0" 202 Accepted 2026-03-28 12:48:53,709 - lava_callback - INFO - Starting async job submit for node 69c52b2ef63ce05331e2b7a7 2026-03-28 12:48:53,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:53,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52b2ef63ce05331e2b7a7 HTTP/1.1" 200 None 2026-03-28 12:48:53,725 - lava_callback - DEBUG - Processing node 69c52b2ef63ce05331e2b7a7: baseline-arm-clabbe 2026-03-28 12:48:53,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52b2ef63ce05331e2b7a7 2026-03-28 12:48:53,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:53,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52b2ef63ce05331e2b7a7 HTTP/1.1" 200 None 2026-03-28 12:48:53,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:53,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:48:53,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:48:53,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52b2ef63ce05331e2b7a7 HTTP/1.1" 200 None 2026-03-28 12:48:53,799 - lava_callback - ERROR - Error processing callback for node 69c52b2ef63ce05331e2b7a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44950 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:52:13,847 - lava_callback - INFO - Received callback for node 69c52bf9f63ce05331e2b7ea INFO: 172.20.0.1:40124 - "POST /node/69c52bf9f63ce05331e2b7ea HTTP/1.0" 202 Accepted 2026-03-28 12:52:13,852 - lava_callback - INFO - Starting async job submit for node 69c52bf9f63ce05331e2b7ea 2026-03-28 12:52:13,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:52:13,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52bf9f63ce05331e2b7ea HTTP/1.1" 200 None 2026-03-28 12:52:13,869 - lava_callback - DEBUG - Processing node 69c52bf9f63ce05331e2b7ea: baseline-arm-clabbe 2026-03-28 12:52:13,869 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52bf9f63ce05331e2b7ea 2026-03-28 12:52:13,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:52:13,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52bf9f63ce05331e2b7ea HTTP/1.1" 200 None 2026-03-28 12:52:13,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:52:13,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 12:52:13,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:52:13,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52bf9f63ce05331e2b7ea HTTP/1.1" 200 None 2026-03-28 12:52:13,921 - lava_callback - ERROR - Error processing callback for node 69c52bf9f63ce05331e2b7ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39128 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:53:13,857 - lava_callback - INFO - Received callback for node 69c52c25f63ce05331e2b7ff INFO: 172.20.0.1:44250 - "POST /node/69c52c25f63ce05331e2b7ff HTTP/1.0" 202 Accepted 2026-03-28 12:53:13,862 - lava_callback - INFO - Starting async job submit for node 69c52c25f63ce05331e2b7ff 2026-03-28 12:53:13,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:13,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c25f63ce05331e2b7ff HTTP/1.1" 200 None 2026-03-28 12:53:13,878 - lava_callback - DEBUG - Processing node 69c52c25f63ce05331e2b7ff: baseline-arm-clabbe 2026-03-28 12:53:13,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52c25f63ce05331e2b7ff 2026-03-28 12:53:13,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:13,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c25f63ce05331e2b7ff HTTP/1.1" 200 None 2026-03-28 12:53:13,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:13,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:53:13,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:13,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52c25f63ce05331e2b7ff HTTP/1.1" 200 None 2026-03-28 12:53:13,933 - lava_callback - ERROR - Error processing callback for node 69c52c25f63ce05331e2b7ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44584 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:53:53,881 - lava_callback - INFO - Received callback for node 69c52c4cf63ce05331e2b811 INFO: 172.20.0.1:56572 - "POST /node/69c52c4cf63ce05331e2b811 HTTP/1.0" 202 Accepted 2026-03-28 12:53:53,886 - lava_callback - INFO - Starting async job submit for node 69c52c4cf63ce05331e2b811 2026-03-28 12:53:53,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:53,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c4cf63ce05331e2b811 HTTP/1.1" 200 None 2026-03-28 12:53:53,925 - lava_callback - DEBUG - Processing node 69c52c4cf63ce05331e2b811: baseline-arm-clabbe 2026-03-28 12:53:53,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52c4cf63ce05331e2b811 2026-03-28 12:53:53,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:53,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c4cf63ce05331e2b811 HTTP/1.1" 200 None 2026-03-28 12:53:53,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:53,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 12:53:53,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:53:54,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52c4cf63ce05331e2b811 HTTP/1.1" 200 None 2026-03-28 12:53:54,020 - lava_callback - ERROR - Error processing callback for node 69c52c4cf63ce05331e2b811: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58412 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:54:33,923 - lava_callback - INFO - Received callback for node 69c52c74f63ce05331e2b820 INFO: 172.20.0.1:34704 - "POST /node/69c52c74f63ce05331e2b820 HTTP/1.0" 202 Accepted 2026-03-28 12:54:33,928 - lava_callback - INFO - Starting async job submit for node 69c52c74f63ce05331e2b820 2026-03-28 12:54:33,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:54:33,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c74f63ce05331e2b820 HTTP/1.1" 200 None 2026-03-28 12:54:33,944 - lava_callback - DEBUG - Processing node 69c52c74f63ce05331e2b820: baseline-arm-clabbe 2026-03-28 12:54:33,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52c74f63ce05331e2b820 2026-03-28 12:54:33,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:54:33,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c74f63ce05331e2b820 HTTP/1.1" 200 None 2026-03-28 12:54:33,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:54:33,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 12:54:33,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:54:33,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52c74f63ce05331e2b820 HTTP/1.1" 200 None 2026-03-28 12:54:34,005 - lava_callback - ERROR - Error processing callback for node 69c52c74f63ce05331e2b820: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39764 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:55:13,857 - lava_callback - INFO - Received callback for node 69c52c9df63ce05331e2b836 INFO: 172.20.0.1:36808 - "POST /node/69c52c9df63ce05331e2b836 HTTP/1.0" 202 Accepted 2026-03-28 12:55:13,862 - lava_callback - INFO - Starting async job submit for node 69c52c9df63ce05331e2b836 2026-03-28 12:55:13,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:13,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c9df63ce05331e2b836 HTTP/1.1" 200 None 2026-03-28 12:55:13,878 - lava_callback - DEBUG - Processing node 69c52c9df63ce05331e2b836: baseline-arm-clabbe 2026-03-28 12:55:13,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52c9df63ce05331e2b836 2026-03-28 12:55:13,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:13,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52c9df63ce05331e2b836 HTTP/1.1" 200 None 2026-03-28 12:55:13,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:13,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 12:55:13,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:13,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52c9df63ce05331e2b836 HTTP/1.1" 200 None 2026-03-28 12:55:13,938 - lava_callback - ERROR - Error processing callback for node 69c52c9df63ce05331e2b836: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46122 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:55:53,950 - lava_callback - INFO - Received callback for node 69c52cc5f63ce05331e2b847 INFO: 172.20.0.1:49410 - "POST /node/69c52cc5f63ce05331e2b847 HTTP/1.0" 202 Accepted 2026-03-28 12:55:53,956 - lava_callback - INFO - Starting async job submit for node 69c52cc5f63ce05331e2b847 2026-03-28 12:55:53,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:53,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52cc5f63ce05331e2b847 HTTP/1.1" 200 None 2026-03-28 12:55:53,972 - lava_callback - DEBUG - Processing node 69c52cc5f63ce05331e2b847: baseline-arm-clabbe 2026-03-28 12:55:53,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52cc5f63ce05331e2b847 2026-03-28 12:55:53,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:53,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52cc5f63ce05331e2b847 HTTP/1.1" 200 None 2026-03-28 12:55:53,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:54,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 12:55:54,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:55:54,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52cc5f63ce05331e2b847 HTTP/1.1" 200 None 2026-03-28 12:55:54,051 - lava_callback - ERROR - Error processing callback for node 69c52cc5f63ce05331e2b847: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45730 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:56:33,963 - lava_callback - INFO - Received callback for node 69c52cecf63ce05331e2b856 INFO: 172.20.0.1:56320 - "POST /node/69c52cecf63ce05331e2b856 HTTP/1.0" 202 Accepted 2026-03-28 12:56:33,968 - lava_callback - INFO - Starting async job submit for node 69c52cecf63ce05331e2b856 2026-03-28 12:56:33,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:56:33,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52cecf63ce05331e2b856 HTTP/1.1" 200 None 2026-03-28 12:56:33,990 - lava_callback - DEBUG - Processing node 69c52cecf63ce05331e2b856: baseline-arm-clabbe 2026-03-28 12:56:33,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52cecf63ce05331e2b856 2026-03-28 12:56:33,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:56:34,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52cecf63ce05331e2b856 HTTP/1.1" 200 None 2026-03-28 12:56:34,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:56:34,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:56:34,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:56:34,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52cecf63ce05331e2b856 HTTP/1.1" 200 None 2026-03-28 12:56:34,043 - lava_callback - ERROR - Error processing callback for node 69c52cecf63ce05331e2b856: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56682 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:57:14,013 - lava_callback - INFO - Received callback for node 69c52d15f63ce05331e2b867 INFO: 172.20.0.1:58856 - "POST /node/69c52d15f63ce05331e2b867 HTTP/1.0" 202 Accepted 2026-03-28 12:57:14,019 - lava_callback - INFO - Starting async job submit for node 69c52d15f63ce05331e2b867 2026-03-28 12:57:14,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:57:14,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52d15f63ce05331e2b867 HTTP/1.1" 200 None 2026-03-28 12:57:14,040 - lava_callback - DEBUG - Processing node 69c52d15f63ce05331e2b867: baseline-arm-clabbe 2026-03-28 12:57:14,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52d15f63ce05331e2b867 2026-03-28 12:57:14,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:57:14,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52d15f63ce05331e2b867 HTTP/1.1" 200 None 2026-03-28 12:57:14,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:57:14,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:57:14,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:57:14,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52d15f63ce05331e2b867 HTTP/1.1" 200 None 2026-03-28 12:57:14,093 - lava_callback - ERROR - Error processing callback for node 69c52d15f63ce05331e2b867: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:58:14,052 - lava_callback - INFO - Received callback for node 69c52d52f63ce05331e2b87b INFO: 172.20.0.1:35198 - "POST /node/69c52d52f63ce05331e2b87b HTTP/1.0" 202 Accepted 2026-03-28 12:58:14,057 - lava_callback - INFO - Starting async job submit for node 69c52d52f63ce05331e2b87b 2026-03-28 12:58:14,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:14,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52d52f63ce05331e2b87b HTTP/1.1" 200 None 2026-03-28 12:58:14,073 - lava_callback - DEBUG - Processing node 69c52d52f63ce05331e2b87b: baseline-arm-clabbe 2026-03-28 12:58:14,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52d52f63ce05331e2b87b 2026-03-28 12:58:14,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:14,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52d52f63ce05331e2b87b HTTP/1.1" 200 None 2026-03-28 12:58:14,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:14,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:58:14,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:14,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52d52f63ce05331e2b87b HTTP/1.1" 200 None 2026-03-28 12:58:14,138 - lava_callback - ERROR - Error processing callback for node 69c52d52f63ce05331e2b87b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35644 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:58:54,055 - lava_callback - INFO - Received callback for node 69c52d79f63ce05331e2b88b INFO: 172.20.0.1:53690 - "POST /node/69c52d79f63ce05331e2b88b HTTP/1.0" 202 Accepted 2026-03-28 12:58:54,060 - lava_callback - INFO - Starting async job submit for node 69c52d79f63ce05331e2b88b 2026-03-28 12:58:54,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:54,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52d79f63ce05331e2b88b HTTP/1.1" 200 None 2026-03-28 12:58:54,077 - lava_callback - DEBUG - Processing node 69c52d79f63ce05331e2b88b: baseline-arm-clabbe 2026-03-28 12:58:54,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52d79f63ce05331e2b88b 2026-03-28 12:58:54,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:54,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52d79f63ce05331e2b88b HTTP/1.1" 200 None 2026-03-28 12:58:54,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:54,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:58:54,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:58:54,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52d79f63ce05331e2b88b HTTP/1.1" 200 None 2026-03-28 12:58:54,142 - lava_callback - ERROR - Error processing callback for node 69c52d79f63ce05331e2b88b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49786 - "GET / HTTP/1.0" 200 OK 2026-03-28 12:59:34,136 - lava_callback - INFO - Received callback for node 69c52da0f63ce05331e2b89a INFO: 172.20.0.1:56166 - "POST /node/69c52da0f63ce05331e2b89a HTTP/1.0" 202 Accepted 2026-03-28 12:59:34,142 - lava_callback - INFO - Starting async job submit for node 69c52da0f63ce05331e2b89a 2026-03-28 12:59:34,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:59:34,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52da0f63ce05331e2b89a HTTP/1.1" 200 None 2026-03-28 12:59:34,158 - lava_callback - DEBUG - Processing node 69c52da0f63ce05331e2b89a: baseline-arm-clabbe 2026-03-28 12:59:34,159 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52da0f63ce05331e2b89a 2026-03-28 12:59:34,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:59:34,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52da0f63ce05331e2b89a HTTP/1.1" 200 None 2026-03-28 12:59:34,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:59:34,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 12:59:34,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 12:59:34,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52da0f63ce05331e2b89a HTTP/1.1" 200 None 2026-03-28 12:59:34,225 - lava_callback - ERROR - Error processing callback for node 69c52da0f63ce05331e2b89a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38576 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:00:14,200 - lava_callback - INFO - Received callback for node 69c52dcaf63ce05331e2b8ae INFO: 172.20.0.1:58040 - "POST /node/69c52dcaf63ce05331e2b8ae HTTP/1.0" 202 Accepted 2026-03-28 13:00:14,205 - lava_callback - INFO - Starting async job submit for node 69c52dcaf63ce05331e2b8ae 2026-03-28 13:00:14,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:14,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52dcaf63ce05331e2b8ae HTTP/1.1" 200 None 2026-03-28 13:00:14,221 - lava_callback - DEBUG - Processing node 69c52dcaf63ce05331e2b8ae: baseline-arm-clabbe 2026-03-28 13:00:14,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52dcaf63ce05331e2b8ae 2026-03-28 13:00:14,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:14,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52dcaf63ce05331e2b8ae HTTP/1.1" 200 None 2026-03-28 13:00:14,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:14,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 13:00:14,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:14,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52dcaf63ce05331e2b8ae HTTP/1.1" 200 None 2026-03-28 13:00:14,283 - lava_callback - ERROR - Error processing callback for node 69c52dcaf63ce05331e2b8ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44840 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:00:54,135 - lava_callback - INFO - Received callback for node 69c52df1f63ce05331e2b8be INFO: 172.20.0.1:46364 - "POST /node/69c52df1f63ce05331e2b8be HTTP/1.0" 202 Accepted 2026-03-28 13:00:54,140 - lava_callback - INFO - Starting async job submit for node 69c52df1f63ce05331e2b8be 2026-03-28 13:00:54,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:54,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52df1f63ce05331e2b8be HTTP/1.1" 200 None 2026-03-28 13:00:54,157 - lava_callback - DEBUG - Processing node 69c52df1f63ce05331e2b8be: baseline-arm-clabbe 2026-03-28 13:00:54,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52df1f63ce05331e2b8be 2026-03-28 13:00:54,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:54,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52df1f63ce05331e2b8be HTTP/1.1" 200 None 2026-03-28 13:00:54,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:54,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 13:00:54,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:00:54,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52df1f63ce05331e2b8be HTTP/1.1" 200 None 2026-03-28 13:00:54,215 - lava_callback - ERROR - Error processing callback for node 69c52df1f63ce05331e2b8be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37742 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:01:34,142 - lava_callback - INFO - Received callback for node 69c52e19f63ce05331e2b8d0 INFO: 172.20.0.1:57962 - "POST /node/69c52e19f63ce05331e2b8d0 HTTP/1.0" 202 Accepted 2026-03-28 13:01:34,148 - lava_callback - INFO - Starting async job submit for node 69c52e19f63ce05331e2b8d0 2026-03-28 13:01:34,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:01:34,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52e19f63ce05331e2b8d0 HTTP/1.1" 200 None 2026-03-28 13:01:34,164 - lava_callback - DEBUG - Processing node 69c52e19f63ce05331e2b8d0: baseline-arm-clabbe 2026-03-28 13:01:34,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52e19f63ce05331e2b8d0 2026-03-28 13:01:34,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:01:34,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52e19f63ce05331e2b8d0 HTTP/1.1" 200 None 2026-03-28 13:01:34,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:01:34,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 13:01:34,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:01:34,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52e19f63ce05331e2b8d0 HTTP/1.1" 200 None 2026-03-28 13:01:34,236 - lava_callback - ERROR - Error processing callback for node 69c52e19f63ce05331e2b8d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:02:14,202 - lava_callback - INFO - Received callback for node 69c52e40f63ce05331e2b8e0 INFO: 172.20.0.1:32974 - "POST /node/69c52e40f63ce05331e2b8e0 HTTP/1.0" 202 Accepted 2026-03-28 13:02:14,207 - lava_callback - INFO - Starting async job submit for node 69c52e40f63ce05331e2b8e0 2026-03-28 13:02:14,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:02:14,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52e40f63ce05331e2b8e0 HTTP/1.1" 200 None 2026-03-28 13:02:14,249 - lava_callback - DEBUG - Processing node 69c52e40f63ce05331e2b8e0: baseline-arm-clabbe 2026-03-28 13:02:14,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c52e40f63ce05331e2b8e0 2026-03-28 13:02:14,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:02:14,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c52e40f63ce05331e2b8e0 HTTP/1.1" 200 None 2026-03-28 13:02:14,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:02:14,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 13:02:14,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:02:14,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c52e40f63ce05331e2b8e0 HTTP/1.1" 200 None 2026-03-28 13:02:14,318 - lava_callback - ERROR - Error processing callback for node 69c52e40f63ce05331e2b8e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57716 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:11:06,677 - lava_callback - INFO - Received callback for node 69c53068f63ce05331e2b91d INFO: 172.20.0.1:57718 - "POST /node/69c53068f63ce05331e2b91d HTTP/1.0" 202 Accepted 2026-03-28 13:11:06,683 - lava_callback - INFO - Starting async job submit for node 69c53068f63ce05331e2b91d 2026-03-28 13:11:06,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:11:06,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53068f63ce05331e2b91d HTTP/1.1" 200 None 2026-03-28 13:11:06,699 - lava_callback - DEBUG - Processing node 69c53068f63ce05331e2b91d: baseline-arm64-baylibre 2026-03-28 13:11:06,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53068f63ce05331e2b91d 2026-03-28 13:11:06,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:11:06,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53068f63ce05331e2b91d HTTP/1.1" 200 None 2026-03-28 13:11:06,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:11:06,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:11:06,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:11:06,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53068f63ce05331e2b91d HTTP/1.1" 200 None 2026-03-28 13:11:06,760 - lava_callback - ERROR - Error processing callback for node 69c53068f63ce05331e2b91d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:13:06,799 - lava_callback - INFO - Received callback for node 69c530def63ce05331e2b928 INFO: 172.20.0.1:33386 - "POST /node/69c530def63ce05331e2b928 HTTP/1.0" 202 Accepted 2026-03-28 13:13:06,805 - lava_callback - INFO - Starting async job submit for node 69c530def63ce05331e2b928 2026-03-28 13:13:06,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:13:06,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c530def63ce05331e2b928 HTTP/1.1" 200 None 2026-03-28 13:13:06,831 - lava_callback - DEBUG - Processing node 69c530def63ce05331e2b928: baseline-arm64-baylibre 2026-03-28 13:13:06,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c530def63ce05331e2b928 2026-03-28 13:13:06,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:13:06,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c530def63ce05331e2b928 HTTP/1.1" 200 None 2026-03-28 13:13:06,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:13:06,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:13:06,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:13:06,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c530def63ce05331e2b928 HTTP/1.1" 200 None 2026-03-28 13:13:06,925 - lava_callback - ERROR - Error processing callback for node 69c530def63ce05331e2b928: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55398 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:14:26,864 - lava_callback - INFO - Received callback for node 69c5312ef63ce05331e2b930 INFO: 172.20.0.1:55402 - "POST /node/69c5312ef63ce05331e2b930 HTTP/1.0" 202 Accepted 2026-03-28 13:14:26,869 - lava_callback - INFO - Starting async job submit for node 69c5312ef63ce05331e2b930 2026-03-28 13:14:26,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:14:26,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5312ef63ce05331e2b930 HTTP/1.1" 200 None 2026-03-28 13:14:26,908 - lava_callback - DEBUG - Processing node 69c5312ef63ce05331e2b930: baseline-arm64-baylibre 2026-03-28 13:14:26,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5312ef63ce05331e2b930 2026-03-28 13:14:26,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:14:26,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5312ef63ce05331e2b930 HTTP/1.1" 200 None 2026-03-28 13:14:26,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:14:26,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:14:26,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:14:27,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5312ef63ce05331e2b930 HTTP/1.1" 200 None 2026-03-28 13:14:27,007 - lava_callback - ERROR - Error processing callback for node 69c5312ef63ce05331e2b930: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32928 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:16:06,984 - lava_callback - INFO - Received callback for node 69c53194f63ce05331e2b940 INFO: 172.20.0.1:32930 - "POST /node/69c53194f63ce05331e2b940 HTTP/1.0" 202 Accepted 2026-03-28 13:16:06,989 - lava_callback - INFO - Starting async job submit for node 69c53194f63ce05331e2b940 2026-03-28 13:16:06,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:16:07,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53194f63ce05331e2b940 HTTP/1.1" 200 None 2026-03-28 13:16:07,025 - lava_callback - DEBUG - Processing node 69c53194f63ce05331e2b940: baseline-arm64-baylibre 2026-03-28 13:16:07,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53194f63ce05331e2b940 2026-03-28 13:16:07,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:16:07,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53194f63ce05331e2b940 HTTP/1.1" 200 None 2026-03-28 13:16:07,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:16:07,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:16:07,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:16:07,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53194f63ce05331e2b940 HTTP/1.1" 200 None 2026-03-28 13:16:07,109 - lava_callback - ERROR - Error processing callback for node 69c53194f63ce05331e2b940: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:17:27,065 - lava_callback - INFO - Received callback for node 69c531e1f63ce05331e2b94a INFO: 172.20.0.1:42782 - "POST /node/69c531e1f63ce05331e2b94a HTTP/1.0" 202 Accepted 2026-03-28 13:17:27,070 - lava_callback - INFO - Starting async job submit for node 69c531e1f63ce05331e2b94a 2026-03-28 13:17:27,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:17:27,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c531e1f63ce05331e2b94a HTTP/1.1" 200 None 2026-03-28 13:17:27,089 - lava_callback - DEBUG - Processing node 69c531e1f63ce05331e2b94a: baseline-arm64-baylibre 2026-03-28 13:17:27,089 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c531e1f63ce05331e2b94a 2026-03-28 13:17:27,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:17:27,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c531e1f63ce05331e2b94a HTTP/1.1" 200 None 2026-03-28 13:17:27,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:17:27,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:17:27,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:17:27,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c531e1f63ce05331e2b94a HTTP/1.1" 200 None 2026-03-28 13:17:27,159 - lava_callback - ERROR - Error processing callback for node 69c531e1f63ce05331e2b94a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:18:47,113 - lava_callback - INFO - Received callback for node 69c53232f63ce05331e2b95b INFO: 172.20.0.1:51190 - "POST /node/69c53232f63ce05331e2b95b HTTP/1.0" 202 Accepted 2026-03-28 13:18:47,122 - lava_callback - INFO - Starting async job submit for node 69c53232f63ce05331e2b95b 2026-03-28 13:18:47,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:18:47,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53232f63ce05331e2b95b HTTP/1.1" 200 None 2026-03-28 13:18:47,145 - lava_callback - DEBUG - Processing node 69c53232f63ce05331e2b95b: baseline-arm64-baylibre 2026-03-28 13:18:47,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53232f63ce05331e2b95b 2026-03-28 13:18:47,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:18:47,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53232f63ce05331e2b95b HTTP/1.1" 200 None 2026-03-28 13:18:47,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:18:47,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:18:47,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:18:47,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53232f63ce05331e2b95b HTTP/1.1" 200 None 2026-03-28 13:18:47,203 - lava_callback - ERROR - Error processing callback for node 69c53232f63ce05331e2b95b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38962 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:20:07,177 - lava_callback - INFO - Received callback for node 69c5327ff63ce05331e2b96b INFO: 172.20.0.1:38968 - "POST /node/69c5327ff63ce05331e2b96b HTTP/1.0" 202 Accepted 2026-03-28 13:20:07,182 - lava_callback - INFO - Starting async job submit for node 69c5327ff63ce05331e2b96b 2026-03-28 13:20:07,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:20:07,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5327ff63ce05331e2b96b HTTP/1.1" 200 None 2026-03-28 13:20:07,199 - lava_callback - DEBUG - Processing node 69c5327ff63ce05331e2b96b: baseline-arm64-baylibre 2026-03-28 13:20:07,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5327ff63ce05331e2b96b 2026-03-28 13:20:07,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:20:07,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5327ff63ce05331e2b96b HTTP/1.1" 200 None 2026-03-28 13:20:07,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:20:07,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:20:07,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:20:07,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5327ff63ce05331e2b96b HTTP/1.1" 200 None 2026-03-28 13:20:07,254 - lava_callback - ERROR - Error processing callback for node 69c5327ff63ce05331e2b96b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53762 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:22:47,490 - lava_callback - INFO - Received callback for node 69c5331bf63ce05331e2b982 INFO: 172.20.0.1:53764 - "POST /node/69c5331bf63ce05331e2b982 HTTP/1.0" 202 Accepted 2026-03-28 13:22:47,495 - lava_callback - INFO - Starting async job submit for node 69c5331bf63ce05331e2b982 2026-03-28 13:22:47,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:22:47,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5331bf63ce05331e2b982 HTTP/1.1" 200 None 2026-03-28 13:22:47,520 - lava_callback - DEBUG - Processing node 69c5331bf63ce05331e2b982: baseline-arm64-baylibre 2026-03-28 13:22:47,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5331bf63ce05331e2b982 2026-03-28 13:22:47,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:22:47,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5331bf63ce05331e2b982 HTTP/1.1" 200 None 2026-03-28 13:22:47,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:22:47,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:22:47,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:22:47,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5331bf63ce05331e2b982 HTTP/1.1" 200 None 2026-03-28 13:22:47,587 - lava_callback - ERROR - Error processing callback for node 69c5331bf63ce05331e2b982: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36254 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:25:27,485 - lava_callback - INFO - Received callback for node 69c533b9f63ce05331e2b99b INFO: 172.20.0.1:36264 - "POST /node/69c533b9f63ce05331e2b99b HTTP/1.0" 202 Accepted 2026-03-28 13:25:27,490 - lava_callback - INFO - Starting async job submit for node 69c533b9f63ce05331e2b99b 2026-03-28 13:25:27,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:25:27,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c533b9f63ce05331e2b99b HTTP/1.1" 200 None 2026-03-28 13:25:27,507 - lava_callback - DEBUG - Processing node 69c533b9f63ce05331e2b99b: baseline-arm64-baylibre 2026-03-28 13:25:27,508 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c533b9f63ce05331e2b99b 2026-03-28 13:25:27,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:25:27,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c533b9f63ce05331e2b99b HTTP/1.1" 200 None 2026-03-28 13:25:27,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:25:27,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:25:27,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:25:27,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c533b9f63ce05331e2b99b HTTP/1.1" 200 None 2026-03-28 13:25:27,559 - lava_callback - ERROR - Error processing callback for node 69c533b9f63ce05331e2b99b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:28:07,623 - lava_callback - INFO - Received callback for node 69c53457f63ce05331e2b9b1 INFO: 172.20.0.1:38680 - "POST /node/69c53457f63ce05331e2b9b1 HTTP/1.0" 202 Accepted 2026-03-28 13:28:07,628 - lava_callback - INFO - Starting async job submit for node 69c53457f63ce05331e2b9b1 2026-03-28 13:28:07,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:28:07,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53457f63ce05331e2b9b1 HTTP/1.1" 200 None 2026-03-28 13:28:07,644 - lava_callback - DEBUG - Processing node 69c53457f63ce05331e2b9b1: baseline-arm64-baylibre 2026-03-28 13:28:07,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53457f63ce05331e2b9b1 2026-03-28 13:28:07,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:28:07,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53457f63ce05331e2b9b1 HTTP/1.1" 200 None 2026-03-28 13:28:07,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:28:07,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:28:07,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:28:07,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53457f63ce05331e2b9b1 HTTP/1.1" 200 None 2026-03-28 13:28:07,723 - lava_callback - ERROR - Error processing callback for node 69c53457f63ce05331e2b9b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35558 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:32:08,324 - lava_callback - INFO - Received callback for node 69c53545f63ce05331e2b9d4 INFO: 172.20.0.1:35572 - "POST /node/69c53545f63ce05331e2b9d4 HTTP/1.0" 202 Accepted 2026-03-28 13:32:08,329 - lava_callback - INFO - Starting async job submit for node 69c53545f63ce05331e2b9d4 2026-03-28 13:32:08,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:32:08,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53545f63ce05331e2b9d4 HTTP/1.1" 200 None 2026-03-28 13:32:08,349 - lava_callback - DEBUG - Processing node 69c53545f63ce05331e2b9d4: baseline-arm64-baylibre 2026-03-28 13:32:08,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53545f63ce05331e2b9d4 2026-03-28 13:32:08,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:32:08,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53545f63ce05331e2b9d4 HTTP/1.1" 200 None 2026-03-28 13:32:08,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:32:08,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:32:08,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:32:08,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53545f63ce05331e2b9d4 HTTP/1.1" 200 None 2026-03-28 13:32:08,416 - lava_callback - ERROR - Error processing callback for node 69c53545f63ce05331e2b9d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40600 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:33:27,971 - lava_callback - INFO - Received callback for node 69c53594f63ce05331e2b9e7 INFO: 172.20.0.1:40606 - "POST /node/69c53594f63ce05331e2b9e7 HTTP/1.0" 202 Accepted 2026-03-28 13:33:27,977 - lava_callback - INFO - Starting async job submit for node 69c53594f63ce05331e2b9e7 2026-03-28 13:33:27,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:33:27,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53594f63ce05331e2b9e7 HTTP/1.1" 200 None 2026-03-28 13:33:27,992 - lava_callback - DEBUG - Processing node 69c53594f63ce05331e2b9e7: baseline-arm64-baylibre 2026-03-28 13:33:27,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53594f63ce05331e2b9e7 2026-03-28 13:33:27,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:33:28,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53594f63ce05331e2b9e7 HTTP/1.1" 200 None 2026-03-28 13:33:28,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:33:28,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:33:28,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:33:28,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53594f63ce05331e2b9e7 HTTP/1.1" 200 None 2026-03-28 13:33:28,064 - lava_callback - ERROR - Error processing callback for node 69c53594f63ce05331e2b9e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33708 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:34:27,982 - lava_callback - INFO - Received callback for node 69c535dff63ce05331e2b9f1 INFO: 172.20.0.1:33722 - "POST /node/69c535dff63ce05331e2b9f1 HTTP/1.0" 202 Accepted 2026-03-28 13:34:27,988 - lava_callback - INFO - Starting async job submit for node 69c535dff63ce05331e2b9f1 2026-03-28 13:34:27,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:34:28,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c535dff63ce05331e2b9f1 HTTP/1.1" 200 None 2026-03-28 13:34:28,004 - lava_callback - DEBUG - Processing node 69c535dff63ce05331e2b9f1: baseline-arm64-baylibre 2026-03-28 13:34:28,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c535dff63ce05331e2b9f1 2026-03-28 13:34:28,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:34:28,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c535dff63ce05331e2b9f1 HTTP/1.1" 200 None 2026-03-28 13:34:28,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:34:28,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:34:28,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:34:28,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c535dff63ce05331e2b9f1 HTTP/1.1" 200 None 2026-03-28 13:34:28,066 - lava_callback - ERROR - Error processing callback for node 69c535dff63ce05331e2b9f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60830 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:36:08,142 - lava_callback - INFO - Received callback for node 69c53634f63ce05331e2b9fd INFO: 172.20.0.1:60838 - "POST /node/69c53634f63ce05331e2b9fd HTTP/1.0" 202 Accepted 2026-03-28 13:36:08,148 - lava_callback - INFO - Starting async job submit for node 69c53634f63ce05331e2b9fd 2026-03-28 13:36:08,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:36:08,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53634f63ce05331e2b9fd HTTP/1.1" 200 None 2026-03-28 13:36:08,163 - lava_callback - DEBUG - Processing node 69c53634f63ce05331e2b9fd: baseline-arm64-baylibre 2026-03-28 13:36:08,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53634f63ce05331e2b9fd 2026-03-28 13:36:08,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:36:08,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53634f63ce05331e2b9fd HTTP/1.1" 200 None 2026-03-28 13:36:08,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:36:08,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 13:36:08,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:36:08,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53634f63ce05331e2b9fd HTTP/1.1" 200 None 2026-03-28 13:36:08,257 - lava_callback - ERROR - Error processing callback for node 69c53634f63ce05331e2b9fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39488 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 13:37:15,771 - lava_callback - INFO - Received callback for node 69c53688f63ce05331e2ba0f INFO: 172.20.0.1:39500 - "POST /node/69c53688f63ce05331e2ba0f HTTP/1.0" 202 Accepted 2026-03-28 13:37:15,777 - lava_callback - INFO - Starting async job submit for node 69c53688f63ce05331e2ba0f 2026-03-28 13:37:15,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:37:15,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53688f63ce05331e2ba0f HTTP/1.1" 200 None 2026-03-28 13:37:15,801 - lava_callback - DEBUG - Processing node 69c53688f63ce05331e2ba0f: baseline-arm64-clabbe 2026-03-28 13:37:15,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53688f63ce05331e2ba0f 2026-03-28 13:37:15,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:37:15,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53688f63ce05331e2ba0f HTTP/1.1" 200 None 2026-03-28 13:37:15,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:37:15,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 13:37:15,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:37:15,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53688f63ce05331e2ba0f HTTP/1.1" 200 None 2026-03-28 13:37:15,853 - lava_callback - ERROR - Error processing callback for node 69c53688f63ce05331e2ba0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36978 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:38:35,721 - lava_callback - INFO - Received callback for node 69c536d9f63ce05331e2ba1e INFO: 172.20.0.1:43364 - "POST /node/69c536d9f63ce05331e2ba1e HTTP/1.0" 202 Accepted 2026-03-28 13:38:35,726 - lava_callback - INFO - Starting async job submit for node 69c536d9f63ce05331e2ba1e 2026-03-28 13:38:35,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:38:35,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c536d9f63ce05331e2ba1e HTTP/1.1" 200 None 2026-03-28 13:38:35,751 - lava_callback - DEBUG - Processing node 69c536d9f63ce05331e2ba1e: baseline-arm64-clabbe 2026-03-28 13:38:35,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c536d9f63ce05331e2ba1e 2026-03-28 13:38:35,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:38:35,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c536d9f63ce05331e2ba1e HTTP/1.1" 200 None 2026-03-28 13:38:35,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:38:35,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:38:35,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:38:35,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c536d9f63ce05331e2ba1e HTTP/1.1" 200 None 2026-03-28 13:38:35,808 - lava_callback - ERROR - Error processing callback for node 69c536d9f63ce05331e2ba1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59974 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:39:55,786 - lava_callback - INFO - Received callback for node 69c53728f63ce05331e2ba28 INFO: 172.20.0.1:33720 - "POST /node/69c53728f63ce05331e2ba28 HTTP/1.0" 202 Accepted 2026-03-28 13:39:55,792 - lava_callback - INFO - Starting async job submit for node 69c53728f63ce05331e2ba28 2026-03-28 13:39:55,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:39:55,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53728f63ce05331e2ba28 HTTP/1.1" 200 None 2026-03-28 13:39:55,807 - lava_callback - DEBUG - Processing node 69c53728f63ce05331e2ba28: baseline-arm64-clabbe 2026-03-28 13:39:55,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53728f63ce05331e2ba28 2026-03-28 13:39:55,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:39:55,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53728f63ce05331e2ba28 HTTP/1.1" 200 None 2026-03-28 13:39:55,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:39:55,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 13:39:55,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:39:55,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53728f63ce05331e2ba28 HTTP/1.1" 200 None 2026-03-28 13:39:55,860 - lava_callback - ERROR - Error processing callback for node 69c53728f63ce05331e2ba28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48044 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 13:41:15,935 - lava_callback - INFO - Received callback for node 69c53778f63ce05331e2ba38 INFO: 172.20.0.1:48046 - "POST /node/69c53778f63ce05331e2ba38 HTTP/1.0" 202 Accepted 2026-03-28 13:41:15,940 - lava_callback - INFO - Starting async job submit for node 69c53778f63ce05331e2ba38 2026-03-28 13:41:15,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:41:15,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53778f63ce05331e2ba38 HTTP/1.1" 200 None 2026-03-28 13:41:15,962 - lava_callback - DEBUG - Processing node 69c53778f63ce05331e2ba38: baseline-arm64-clabbe 2026-03-28 13:41:15,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53778f63ce05331e2ba38 2026-03-28 13:41:15,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:41:15,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53778f63ce05331e2ba38 HTTP/1.1" 200 None 2026-03-28 13:41:15,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:41:15,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 13:41:15,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:41:16,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53778f63ce05331e2ba38 HTTP/1.1" 200 None 2026-03-28 13:41:16,025 - lava_callback - ERROR - Error processing callback for node 69c53778f63ce05331e2ba38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60518 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:42:36,024 - lava_callback - INFO - Received callback for node 69c537c7f63ce05331e2ba46 INFO: 172.20.0.1:33390 - "POST /node/69c537c7f63ce05331e2ba46 HTTP/1.0" 202 Accepted 2026-03-28 13:42:36,029 - lava_callback - INFO - Starting async job submit for node 69c537c7f63ce05331e2ba46 2026-03-28 13:42:36,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:42:36,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c537c7f63ce05331e2ba46 HTTP/1.1" 200 None 2026-03-28 13:42:36,044 - lava_callback - DEBUG - Processing node 69c537c7f63ce05331e2ba46: baseline-arm64-clabbe 2026-03-28 13:42:36,045 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c537c7f63ce05331e2ba46 2026-03-28 13:42:36,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:42:36,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c537c7f63ce05331e2ba46 HTTP/1.1" 200 None 2026-03-28 13:42:36,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:42:36,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:42:36,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:42:36,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c537c7f63ce05331e2ba46 HTTP/1.1" 200 None 2026-03-28 13:42:36,101 - lava_callback - ERROR - Error processing callback for node 69c537c7f63ce05331e2ba46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38124 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:43:56,068 - lava_callback - INFO - Received callback for node 69c53815f63ce05331e2ba4e INFO: 172.20.0.1:47476 - "POST /node/69c53815f63ce05331e2ba4e HTTP/1.0" 202 Accepted 2026-03-28 13:43:56,074 - lava_callback - INFO - Starting async job submit for node 69c53815f63ce05331e2ba4e 2026-03-28 13:43:56,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:43:56,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53815f63ce05331e2ba4e HTTP/1.1" 200 None 2026-03-28 13:43:56,090 - lava_callback - DEBUG - Processing node 69c53815f63ce05331e2ba4e: baseline-arm64-clabbe 2026-03-28 13:43:56,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53815f63ce05331e2ba4e 2026-03-28 13:43:56,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:43:56,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53815f63ce05331e2ba4e HTTP/1.1" 200 None 2026-03-28 13:43:56,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:43:56,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 13:43:56,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:43:56,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53815f63ce05331e2ba4e HTTP/1.1" 200 None 2026-03-28 13:43:56,143 - lava_callback - ERROR - Error processing callback for node 69c53815f63ce05331e2ba4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49566 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 13:45:16,035 - lava_callback - INFO - Received callback for node 69c53862f63ce05331e2ba5f INFO: 172.20.0.1:49580 - "POST /node/69c53862f63ce05331e2ba5f HTTP/1.0" 202 Accepted 2026-03-28 13:45:16,040 - lava_callback - INFO - Starting async job submit for node 69c53862f63ce05331e2ba5f 2026-03-28 13:45:16,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:45:16,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53862f63ce05331e2ba5f HTTP/1.1" 200 None 2026-03-28 13:45:16,055 - lava_callback - DEBUG - Processing node 69c53862f63ce05331e2ba5f: baseline-arm64-clabbe 2026-03-28 13:45:16,056 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53862f63ce05331e2ba5f 2026-03-28 13:45:16,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:45:16,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53862f63ce05331e2ba5f HTTP/1.1" 200 None 2026-03-28 13:45:16,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:45:16,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:45:16,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:45:16,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53862f63ce05331e2ba5f HTTP/1.1" 200 None 2026-03-28 13:45:16,118 - lava_callback - ERROR - Error processing callback for node 69c53862f63ce05331e2ba5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51290 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:46:36,127 - lava_callback - INFO - Received callback for node 69c538b1f63ce05331e2ba68 INFO: 172.20.0.1:51732 - "POST /node/69c538b1f63ce05331e2ba68 HTTP/1.0" 202 Accepted 2026-03-28 13:46:36,132 - lava_callback - INFO - Starting async job submit for node 69c538b1f63ce05331e2ba68 2026-03-28 13:46:36,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:46:36,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c538b1f63ce05331e2ba68 HTTP/1.1" 200 None 2026-03-28 13:46:36,148 - lava_callback - DEBUG - Processing node 69c538b1f63ce05331e2ba68: baseline-arm64-clabbe 2026-03-28 13:46:36,149 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c538b1f63ce05331e2ba68 2026-03-28 13:46:36,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:46:36,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c538b1f63ce05331e2ba68 HTTP/1.1" 200 None 2026-03-28 13:46:36,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:46:36,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 13:46:36,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:46:36,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c538b1f63ce05331e2ba68 HTTP/1.1" 200 None 2026-03-28 13:46:36,262 - lava_callback - ERROR - Error processing callback for node 69c538b1f63ce05331e2ba68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57950 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:47:56,185 - lava_callback - INFO - Received callback for node 69c53901f63ce05331e2ba74 INFO: 172.20.0.1:38540 - "POST /node/69c53901f63ce05331e2ba74 HTTP/1.0" 202 Accepted 2026-03-28 13:47:56,191 - lava_callback - INFO - Starting async job submit for node 69c53901f63ce05331e2ba74 2026-03-28 13:47:56,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:47:56,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53901f63ce05331e2ba74 HTTP/1.1" 200 None 2026-03-28 13:47:56,207 - lava_callback - DEBUG - Processing node 69c53901f63ce05331e2ba74: baseline-arm64-clabbe 2026-03-28 13:47:56,207 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53901f63ce05331e2ba74 2026-03-28 13:47:56,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:47:56,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53901f63ce05331e2ba74 HTTP/1.1" 200 None 2026-03-28 13:47:56,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:47:56,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 13:47:56,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:47:56,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53901f63ce05331e2ba74 HTTP/1.1" 200 None 2026-03-28 13:47:56,275 - lava_callback - ERROR - Error processing callback for node 69c53901f63ce05331e2ba74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:49:08,865 - lava_callback - INFO - Received callback for node 69c53946f63ce05331e2ba81 INFO: 172.20.0.1:38362 - "POST /node/69c53946f63ce05331e2ba81 HTTP/1.0" 202 Accepted 2026-03-28 13:49:08,870 - lava_callback - INFO - Starting async job submit for node 69c53946f63ce05331e2ba81 2026-03-28 13:49:08,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:49:08,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53946f63ce05331e2ba81 HTTP/1.1" 200 None 2026-03-28 13:49:08,885 - lava_callback - DEBUG - Processing node 69c53946f63ce05331e2ba81: baseline-arm64-baylibre 2026-03-28 13:49:08,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53946f63ce05331e2ba81 2026-03-28 13:49:08,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:49:08,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53946f63ce05331e2ba81 HTTP/1.1" 200 None 2026-03-28 13:49:08,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:49:08,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:49:08,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:49:08,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53946f63ce05331e2ba81 HTTP/1.1" 200 None 2026-03-28 13:49:08,941 - lava_callback - ERROR - Error processing callback for node 69c53946f63ce05331e2ba81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49822 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:50:36,338 - lava_callback - INFO - Received callback for node 69c5399df63ce05331e2ba8f INFO: 172.20.0.1:46110 - "POST /node/69c5399df63ce05331e2ba8f HTTP/1.0" 202 Accepted 2026-03-28 13:50:36,344 - lava_callback - INFO - Starting async job submit for node 69c5399df63ce05331e2ba8f 2026-03-28 13:50:36,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:50:36,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5399df63ce05331e2ba8f HTTP/1.1" 200 None 2026-03-28 13:50:36,360 - lava_callback - DEBUG - Processing node 69c5399df63ce05331e2ba8f: baseline-arm64-clabbe 2026-03-28 13:50:36,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5399df63ce05331e2ba8f 2026-03-28 13:50:36,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:50:36,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5399df63ce05331e2ba8f HTTP/1.1" 200 None 2026-03-28 13:50:36,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:50:36,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:50:36,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:50:36,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5399df63ce05331e2ba8f HTTP/1.1" 200 None 2026-03-28 13:50:36,420 - lava_callback - ERROR - Error processing callback for node 69c5399df63ce05331e2ba8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49170 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:51:56,397 - lava_callback - INFO - Received callback for node 69c539ebf63ce05331e2ba9a INFO: 172.20.0.1:38830 - "POST /node/69c539ebf63ce05331e2ba9a HTTP/1.0" 202 Accepted 2026-03-28 13:51:56,403 - lava_callback - INFO - Starting async job submit for node 69c539ebf63ce05331e2ba9a 2026-03-28 13:51:56,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:51:56,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c539ebf63ce05331e2ba9a HTTP/1.1" 200 None 2026-03-28 13:51:56,419 - lava_callback - DEBUG - Processing node 69c539ebf63ce05331e2ba9a: baseline-arm64-clabbe 2026-03-28 13:51:56,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c539ebf63ce05331e2ba9a 2026-03-28 13:51:56,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:51:56,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c539ebf63ce05331e2ba9a HTTP/1.1" 200 None 2026-03-28 13:51:56,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:51:56,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:51:56,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:51:56,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c539ebf63ce05331e2ba9a HTTP/1.1" 200 None 2026-03-28 13:51:56,487 - lava_callback - ERROR - Error processing callback for node 69c539ebf63ce05331e2ba9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55558 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:53:09,072 - lava_callback - INFO - Received callback for node 69c53a30f63ce05331e2baa2 INFO: 172.20.0.1:55570 - "POST /node/69c53a30f63ce05331e2baa2 HTTP/1.0" 202 Accepted 2026-03-28 13:53:09,078 - lava_callback - INFO - Starting async job submit for node 69c53a30f63ce05331e2baa2 2026-03-28 13:53:09,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:53:09,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53a30f63ce05331e2baa2 HTTP/1.1" 200 None 2026-03-28 13:53:09,097 - lava_callback - DEBUG - Processing node 69c53a30f63ce05331e2baa2: baseline-arm64-baylibre 2026-03-28 13:53:09,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53a30f63ce05331e2baa2 2026-03-28 13:53:09,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:53:09,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53a30f63ce05331e2baa2 HTTP/1.1" 200 None 2026-03-28 13:53:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:53:09,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:53:09,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:53:09,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53a30f63ce05331e2baa2 HTTP/1.1" 200 None 2026-03-28 13:53:09,152 - lava_callback - ERROR - Error processing callback for node 69c53a30f63ce05331e2baa2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35390 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 13:54:16,356 - lava_callback - INFO - Received callback for node 69c53a85f63ce05331e2bab2 INFO: 172.20.0.1:35394 - "POST /node/69c53a85f63ce05331e2bab2 HTTP/1.0" 202 Accepted 2026-03-28 13:54:16,362 - lava_callback - INFO - Starting async job submit for node 69c53a85f63ce05331e2bab2 2026-03-28 13:54:16,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:54:16,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53a85f63ce05331e2bab2 HTTP/1.1" 200 None 2026-03-28 13:54:16,378 - lava_callback - DEBUG - Processing node 69c53a85f63ce05331e2bab2: baseline-arm64-clabbe 2026-03-28 13:54:16,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53a85f63ce05331e2bab2 2026-03-28 13:54:16,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:54:16,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53a85f63ce05331e2bab2 HTTP/1.1" 200 None 2026-03-28 13:54:16,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:54:16,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:54:16,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:54:16,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53a85f63ce05331e2bab2 HTTP/1.1" 200 None 2026-03-28 13:54:16,444 - lava_callback - ERROR - Error processing callback for node 69c53a85f63ce05331e2bab2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47706 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:55:36,528 - lava_callback - INFO - Received callback for node 69c53ad4f63ce05331e2bac1 INFO: 172.20.0.1:47708 - "POST /node/69c53ad4f63ce05331e2bac1 HTTP/1.0" 202 Accepted 2026-03-28 13:55:36,534 - lava_callback - INFO - Starting async job submit for node 69c53ad4f63ce05331e2bac1 2026-03-28 13:55:36,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:55:36,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53ad4f63ce05331e2bac1 HTTP/1.1" 200 None 2026-03-28 13:55:36,550 - lava_callback - DEBUG - Processing node 69c53ad4f63ce05331e2bac1: baseline-arm64-clabbe 2026-03-28 13:55:36,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53ad4f63ce05331e2bac1 2026-03-28 13:55:36,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:55:36,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53ad4f63ce05331e2bac1 HTTP/1.1" 200 None 2026-03-28 13:55:36,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:55:36,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:55:36,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:55:36,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53ad4f63ce05331e2bac1 HTTP/1.1" 200 None 2026-03-28 13:55:36,610 - lava_callback - ERROR - Error processing callback for node 69c53ad4f63ce05331e2bac1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41814 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:56:56,605 - lava_callback - INFO - Received callback for node 69c53b22f63ce05331e2bacb INFO: 172.20.0.1:41826 - "POST /node/69c53b22f63ce05331e2bacb HTTP/1.0" 202 Accepted 2026-03-28 13:56:56,611 - lava_callback - INFO - Starting async job submit for node 69c53b22f63ce05331e2bacb 2026-03-28 13:56:56,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:56:56,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53b22f63ce05331e2bacb HTTP/1.1" 200 None 2026-03-28 13:56:56,627 - lava_callback - DEBUG - Processing node 69c53b22f63ce05331e2bacb: baseline-arm64-clabbe 2026-03-28 13:56:56,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53b22f63ce05331e2bacb 2026-03-28 13:56:56,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:56:56,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53b22f63ce05331e2bacb HTTP/1.1" 200 None 2026-03-28 13:56:56,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:56:56,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:56:56,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:56:56,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53b22f63ce05331e2bacb HTTP/1.1" 200 None 2026-03-28 13:56:56,699 - lava_callback - ERROR - Error processing callback for node 69c53b22f63ce05331e2bacb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47976 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:58:02,245 - lava_callback - INFO - Received callback for node 69c722ac29015d0f9598f83a INFO: 172.20.0.1:44338 - "POST /node/69c722ac29015d0f9598f83a HTTP/1.0" 202 Accepted 2026-03-28 13:58:02,250 - lava_callback - INFO - Starting async job submit for node 69c722ac29015d0f9598f83a 2026-03-28 13:58:02,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:02,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722ac29015d0f9598f83a HTTP/1.1" 200 None 2026-03-28 13:58:02,283 - lava_callback - DEBUG - Processing node 69c722ac29015d0f9598f83a: baseline-x86 2026-03-28 13:58:02,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 13:58:02,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 13:58:02,750 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ac29015d0f9598f83a/log.txt.gz 2026-03-28 13:58:02,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 13:58:03,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 13:58:03,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ac29015d0f9598f83a/lava-logs.yaml 2026-03-28 13:58:03,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 13:58:03,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 13:58:03,353 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ac29015d0f9598f83a/lava_callback.json.gz 2026-03-28 13:58:03,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:03,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722ac29015d0f9598f83a HTTP/1.1" 200 None 2026-03-28 13:58:03,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:03,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 13:58:03,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:03,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722ac29015d0f9598f83a HTTP/1.1" 200 None 2026-03-28 13:58:03,429 - lava_callback - INFO - Completed processing callback for node 69c722ac29015d0f9598f83a INFO: 172.20.0.1:44344 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:58:09,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:09,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57372 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:58:16,626 - lava_callback - INFO - Received callback for node 69c53b72f63ce05331e2badc INFO: 172.20.0.1:57388 - "POST /node/69c53b72f63ce05331e2badc HTTP/1.0" 202 Accepted 2026-03-28 13:58:16,635 - lava_callback - INFO - Starting async job submit for node 69c53b72f63ce05331e2badc 2026-03-28 13:58:16,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:16,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53b72f63ce05331e2badc HTTP/1.1" 200 None 2026-03-28 13:58:16,651 - lava_callback - DEBUG - Processing node 69c53b72f63ce05331e2badc: baseline-arm64-clabbe 2026-03-28 13:58:16,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53b72f63ce05331e2badc 2026-03-28 13:58:16,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:16,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53b72f63ce05331e2badc HTTP/1.1" 200 None 2026-03-28 13:58:16,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:16,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:58:16,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:58:16,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53b72f63ce05331e2badc HTTP/1.1" 200 None 2026-03-28 13:58:16,710 - lava_callback - ERROR - Error processing callback for node 69c53b72f63ce05331e2badc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37928 - "GET / HTTP/1.0" 200 OK 2026-03-28 13:59:29,527 - lava_callback - INFO - Received callback for node 69c53bb9f63ce05331e2bae3 INFO: 172.20.0.1:37930 - "POST /node/69c53bb9f63ce05331e2bae3 HTTP/1.0" 202 Accepted 2026-03-28 13:59:29,532 - lava_callback - INFO - Starting async job submit for node 69c53bb9f63ce05331e2bae3 2026-03-28 13:59:29,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:59:29,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53bb9f63ce05331e2bae3 HTTP/1.1" 200 None 2026-03-28 13:59:29,549 - lava_callback - DEBUG - Processing node 69c53bb9f63ce05331e2bae3: baseline-arm64-baylibre 2026-03-28 13:59:29,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53bb9f63ce05331e2bae3 2026-03-28 13:59:29,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:59:29,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53bb9f63ce05331e2bae3 HTTP/1.1" 200 None 2026-03-28 13:59:29,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:59:29,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 13:59:29,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 13:59:29,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53bb9f63ce05331e2bae3 HTTP/1.1" 200 None 2026-03-28 13:59:29,623 - lava_callback - ERROR - Error processing callback for node 69c53bb9f63ce05331e2bae3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49230 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:00:49,482 - lava_callback - INFO - Received callback for node 69c53c0af63ce05331e2baf0 INFO: 172.20.0.1:49238 - "POST /node/69c53c0af63ce05331e2baf0 HTTP/1.0" 202 Accepted 2026-03-28 14:00:49,487 - lava_callback - INFO - Starting async job submit for node 69c53c0af63ce05331e2baf0 2026-03-28 14:00:49,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:00:49,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53c0af63ce05331e2baf0 HTTP/1.1" 200 None 2026-03-28 14:00:49,503 - lava_callback - DEBUG - Processing node 69c53c0af63ce05331e2baf0: baseline-arm64-baylibre 2026-03-28 14:00:49,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53c0af63ce05331e2baf0 2026-03-28 14:00:49,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:00:49,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53c0af63ce05331e2baf0 HTTP/1.1" 200 None 2026-03-28 14:00:49,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:00:49,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:00:49,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:00:49,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53c0af63ce05331e2baf0 HTTP/1.1" 200 None 2026-03-28 14:00:49,580 - lava_callback - ERROR - Error processing callback for node 69c53c0af63ce05331e2baf0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:01:17,640 - lava_callback - INFO - Received callback for node 69c7778329015d0f95993016 INFO: 172.20.0.1:46502 - "POST /node/69c7778329015d0f95993016 HTTP/1.0" 202 Accepted 2026-03-28 14:01:17,645 - lava_callback - INFO - Starting async job submit for node 69c7778329015d0f95993016 2026-03-28 14:01:17,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:01:17,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7778329015d0f95993016 HTTP/1.1" 200 None 2026-03-28 14:01:17,683 - lava_callback - DEBUG - Processing node 69c7778329015d0f95993016: baseline-x86 2026-03-28 14:01:17,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:01:18,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:01:18,153 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c7778329015d0f95993016/log.txt.gz 2026-03-28 14:01:18,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:01:18,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:01:18,555 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c7778329015d0f95993016/lava-logs.yaml 2026-03-28 14:01:18,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:01:18,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:01:18,722 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c7778329015d0f95993016/lava_callback.json.gz 2026-03-28 14:01:18,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:01:18,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7778329015d0f95993016 HTTP/1.1" 200 None 2026-03-28 14:01:18,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:01:18,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 14:01:18,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:01:18,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7778329015d0f95993016 HTTP/1.1" 200 None 2026-03-28 14:01:18,798 - lava_callback - INFO - Completed processing callback for node 69c7778329015d0f95993016 INFO: 172.20.0.1:36726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:01:39,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:01:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:02:16,824 - lava_callback - INFO - Received callback for node 69c53c60f63ce05331e2bb00 INFO: 172.20.0.1:46098 - "POST /node/69c53c60f63ce05331e2bb00 HTTP/1.0" 202 Accepted 2026-03-28 14:02:16,829 - lava_callback - INFO - Starting async job submit for node 69c53c60f63ce05331e2bb00 2026-03-28 14:02:16,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:16,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53c60f63ce05331e2bb00 HTTP/1.1" 200 None 2026-03-28 14:02:16,845 - lava_callback - DEBUG - Processing node 69c53c60f63ce05331e2bb00: baseline-arm64-clabbe 2026-03-28 14:02:16,845 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53c60f63ce05331e2bb00 2026-03-28 14:02:16,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:16,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53c60f63ce05331e2bb00 HTTP/1.1" 200 None 2026-03-28 14:02:16,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:16,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:02:16,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:16,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53c60f63ce05331e2bb00 HTTP/1.1" 200 None 2026-03-28 14:02:16,933 - lava_callback - ERROR - Error processing callback for node 69c53c60f63ce05331e2bb00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:02:41,664 - lava_callback - INFO - Received callback for node 69c79654be430e18c5d162db INFO: 172.20.0.1:35356 - "POST /node/69c79654be430e18c5d162db HTTP/1.0" 202 Accepted 2026-03-28 14:02:41,670 - lava_callback - INFO - Starting async job submit for node 69c79654be430e18c5d162db 2026-03-28 14:02:41,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:41,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79654be430e18c5d162db HTTP/1.1" 200 None 2026-03-28 14:02:41,701 - lava_callback - DEBUG - Processing node 69c79654be430e18c5d162db: baseline-x86 2026-03-28 14:02:41,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:02:42,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:02:42,154 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c79654be430e18c5d162db/log.txt.gz 2026-03-28 14:02:42,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:02:42,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:02:42,533 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c79654be430e18c5d162db/lava-logs.yaml 2026-03-28 14:02:42,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:02:42,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:02:42,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c79654be430e18c5d162db/lava_callback.json.gz 2026-03-28 14:02:42,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:42,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c79654be430e18c5d162db HTTP/1.1" 200 None 2026-03-28 14:02:42,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:42,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78ecdbe430e18c5d15a09 HTTP/1.1" 200 None 2026-03-28 14:02:42,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:02:42,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c79654be430e18c5d162db HTTP/1.1" 200 None 2026-03-28 14:02:42,848 - lava_callback - INFO - Completed processing callback for node 69c79654be430e18c5d162db INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:03:09,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:09,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:03:36,858 - lava_callback - INFO - Received callback for node 69c53caef63ce05331e2bb0a INFO: 172.20.0.1:41718 - "POST /node/69c53caef63ce05331e2bb0a HTTP/1.0" 202 Accepted 2026-03-28 14:03:36,864 - lava_callback - INFO - Starting async job submit for node 69c53caef63ce05331e2bb0a 2026-03-28 14:03:36,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:36,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53caef63ce05331e2bb0a HTTP/1.1" 200 None 2026-03-28 14:03:36,880 - lava_callback - DEBUG - Processing node 69c53caef63ce05331e2bb0a: baseline-arm64-clabbe 2026-03-28 14:03:36,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53caef63ce05331e2bb0a 2026-03-28 14:03:36,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:36,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53caef63ce05331e2bb0a HTTP/1.1" 200 None 2026-03-28 14:03:36,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:36,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:03:36,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:36,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53caef63ce05331e2bb0a HTTP/1.1" 200 None 2026-03-28 14:03:36,959 - lava_callback - ERROR - Error processing callback for node 69c53caef63ce05331e2bb0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 14:03:37,744 - lava_callback - INFO - Received callback for node 69c7cc40be430e18c5d16cfd INFO: 172.20.0.1:41728 - "POST /node/69c7cc40be430e18c5d16cfd HTTP/1.0" 202 Accepted 2026-03-28 14:03:37,750 - lava_callback - INFO - Starting async job submit for node 69c7cc40be430e18c5d16cfd 2026-03-28 14:03:37,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:37,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cc40be430e18c5d16cfd HTTP/1.1" 200 None 2026-03-28 14:03:37,783 - lava_callback - DEBUG - Processing node 69c7cc40be430e18c5d16cfd: baseline-x86 2026-03-28 14:03:38,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:03:38,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:03:38,235 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c7cc40be430e18c5d16cfd/log.txt.gz 2026-03-28 14:03:38,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:03:38,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:03:38,623 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c7cc40be430e18c5d16cfd/lava-logs.yaml 2026-03-28 14:03:38,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 14:03:38,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 14:03:38,796 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c7cc40be430e18c5d16cfd/lava_callback.json.gz 2026-03-28 14:03:38,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:38,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7cc40be430e18c5d16cfd HTTP/1.1" 200 None 2026-03-28 14:03:38,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:38,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 14:03:38,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:38,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7cc40be430e18c5d16cfd HTTP/1.1" 200 None 2026-03-28 14:03:38,876 - lava_callback - INFO - Completed processing callback for node 69c7cc40be430e18c5d16cfd 2026-03-28 14:03:39,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:03:39,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:04:49,747 - lava_callback - INFO - Received callback for node 69c53cf8f63ce05331e2bb1a INFO: 172.20.0.1:50950 - "POST /node/69c53cf8f63ce05331e2bb1a HTTP/1.0" 202 Accepted 2026-03-28 14:04:49,753 - lava_callback - INFO - Starting async job submit for node 69c53cf8f63ce05331e2bb1a 2026-03-28 14:04:49,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:04:49,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53cf8f63ce05331e2bb1a HTTP/1.1" 200 None 2026-03-28 14:04:49,775 - lava_callback - DEBUG - Processing node 69c53cf8f63ce05331e2bb1a: baseline-arm64-baylibre 2026-03-28 14:04:49,775 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53cf8f63ce05331e2bb1a 2026-03-28 14:04:49,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:04:49,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53cf8f63ce05331e2bb1a HTTP/1.1" 200 None 2026-03-28 14:04:49,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:04:49,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:04:49,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:04:49,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53cf8f63ce05331e2bb1a HTTP/1.1" 200 None 2026-03-28 14:04:49,853 - lava_callback - ERROR - Error processing callback for node 69c53cf8f63ce05331e2bb1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:06:17,065 - lava_callback - INFO - Received callback for node 69c53d4ef63ce05331e2bb28 INFO: 172.20.0.1:54584 - "POST /node/69c53d4ef63ce05331e2bb28 HTTP/1.0" 202 Accepted 2026-03-28 14:06:17,071 - lava_callback - INFO - Starting async job submit for node 69c53d4ef63ce05331e2bb28 2026-03-28 14:06:17,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:06:17,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53d4ef63ce05331e2bb28 HTTP/1.1" 200 None 2026-03-28 14:06:17,087 - lava_callback - DEBUG - Processing node 69c53d4ef63ce05331e2bb28: baseline-arm64-clabbe 2026-03-28 14:06:17,087 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53d4ef63ce05331e2bb28 2026-03-28 14:06:17,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:06:17,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53d4ef63ce05331e2bb28 HTTP/1.1" 200 None 2026-03-28 14:06:17,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:06:17,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:06:17,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:06:17,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53d4ef63ce05331e2bb28 HTTP/1.1" 200 None 2026-03-28 14:06:17,145 - lava_callback - ERROR - Error processing callback for node 69c53d4ef63ce05331e2bb28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40644 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:07:37,092 - lava_callback - INFO - Received callback for node 69c53da0f63ce05331e2bb31 INFO: 172.20.0.1:40656 - "POST /node/69c53da0f63ce05331e2bb31 HTTP/1.0" 202 Accepted 2026-03-28 14:07:37,104 - lava_callback - INFO - Starting async job submit for node 69c53da0f63ce05331e2bb31 2026-03-28 14:07:37,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:07:37,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53da0f63ce05331e2bb31 HTTP/1.1" 200 None 2026-03-28 14:07:37,138 - lava_callback - DEBUG - Processing node 69c53da0f63ce05331e2bb31: baseline-arm64-clabbe 2026-03-28 14:07:37,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53da0f63ce05331e2bb31 2026-03-28 14:07:37,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:07:37,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53da0f63ce05331e2bb31 HTTP/1.1" 200 None 2026-03-28 14:07:37,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:07:37,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:07:37,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:07:37,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53da0f63ce05331e2bb31 HTTP/1.1" 200 None 2026-03-28 14:07:37,205 - lava_callback - ERROR - Error processing callback for node 69c53da0f63ce05331e2bb31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40788 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:08:57,156 - lava_callback - INFO - Received callback for node 69c53df2f63ce05331e2bb3e INFO: 172.20.0.1:40792 - "POST /node/69c53df2f63ce05331e2bb3e HTTP/1.0" 202 Accepted 2026-03-28 14:08:57,162 - lava_callback - INFO - Starting async job submit for node 69c53df2f63ce05331e2bb3e 2026-03-28 14:08:57,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:08:57,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53df2f63ce05331e2bb3e HTTP/1.1" 200 None 2026-03-28 14:08:57,184 - lava_callback - DEBUG - Processing node 69c53df2f63ce05331e2bb3e: baseline-arm64-clabbe 2026-03-28 14:08:57,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53df2f63ce05331e2bb3e 2026-03-28 14:08:57,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:08:57,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53df2f63ce05331e2bb3e HTTP/1.1" 200 None 2026-03-28 14:08:57,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:08:57,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:08:57,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:08:57,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53df2f63ce05331e2bb3e HTTP/1.1" 200 None 2026-03-28 14:08:57,268 - lava_callback - ERROR - Error processing callback for node 69c53df2f63ce05331e2bb3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51788 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:10:17,193 - lava_callback - INFO - Received callback for node 69c53e40f63ce05331e2bb4c INFO: 172.20.0.1:51796 - "POST /node/69c53e40f63ce05331e2bb4c HTTP/1.0" 202 Accepted 2026-03-28 14:10:17,198 - lava_callback - INFO - Starting async job submit for node 69c53e40f63ce05331e2bb4c 2026-03-28 14:10:17,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:10:17,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53e40f63ce05331e2bb4c HTTP/1.1" 200 None 2026-03-28 14:10:17,216 - lava_callback - DEBUG - Processing node 69c53e40f63ce05331e2bb4c: baseline-arm64-clabbe 2026-03-28 14:10:17,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53e40f63ce05331e2bb4c 2026-03-28 14:10:17,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:10:17,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53e40f63ce05331e2bb4c HTTP/1.1" 200 None 2026-03-28 14:10:17,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:10:17,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:10:17,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:10:17,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53e40f63ce05331e2bb4c HTTP/1.1" 200 None 2026-03-28 14:10:17,267 - lava_callback - ERROR - Error processing callback for node 69c53e40f63ce05331e2bb4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33850 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:11:37,295 - lava_callback - INFO - Received callback for node 69c53e91f63ce05331e2bb5a INFO: 172.20.0.1:33858 - "POST /node/69c53e91f63ce05331e2bb5a HTTP/1.0" 202 Accepted 2026-03-28 14:11:37,305 - lava_callback - INFO - Starting async job submit for node 69c53e91f63ce05331e2bb5a 2026-03-28 14:11:37,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:11:37,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53e91f63ce05331e2bb5a HTTP/1.1" 200 None 2026-03-28 14:11:37,328 - lava_callback - DEBUG - Processing node 69c53e91f63ce05331e2bb5a: baseline-arm64-clabbe 2026-03-28 14:11:37,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53e91f63ce05331e2bb5a 2026-03-28 14:11:37,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:11:37,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53e91f63ce05331e2bb5a HTTP/1.1" 200 None 2026-03-28 14:11:37,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:11:37,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:11:37,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:11:37,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53e91f63ce05331e2bb5a HTTP/1.1" 200 None 2026-03-28 14:11:37,506 - lava_callback - ERROR - Error processing callback for node 69c53e91f63ce05331e2bb5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:12:57,345 - lava_callback - INFO - Received callback for node 69c53ee0f63ce05331e2bb63 INFO: 172.20.0.1:49322 - "POST /node/69c53ee0f63ce05331e2bb63 HTTP/1.0" 202 Accepted 2026-03-28 14:12:57,351 - lava_callback - INFO - Starting async job submit for node 69c53ee0f63ce05331e2bb63 2026-03-28 14:12:57,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:12:57,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53ee0f63ce05331e2bb63 HTTP/1.1" 200 None 2026-03-28 14:12:57,370 - lava_callback - DEBUG - Processing node 69c53ee0f63ce05331e2bb63: baseline-arm64-clabbe 2026-03-28 14:12:57,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53ee0f63ce05331e2bb63 2026-03-28 14:12:57,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:12:57,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53ee0f63ce05331e2bb63 HTTP/1.1" 200 None 2026-03-28 14:12:57,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:12:57,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:12:57,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:12:57,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53ee0f63ce05331e2bb63 HTTP/1.1" 200 None 2026-03-28 14:12:57,424 - lava_callback - ERROR - Error processing callback for node 69c53ee0f63ce05331e2bb63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47484 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:14:17,437 - lava_callback - INFO - Received callback for node 69c53f2ef63ce05331e2bb72 INFO: 172.20.0.1:47496 - "POST /node/69c53f2ef63ce05331e2bb72 HTTP/1.0" 202 Accepted 2026-03-28 14:14:17,442 - lava_callback - INFO - Starting async job submit for node 69c53f2ef63ce05331e2bb72 2026-03-28 14:14:17,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:14:17,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53f2ef63ce05331e2bb72 HTTP/1.1" 200 None 2026-03-28 14:14:17,458 - lava_callback - DEBUG - Processing node 69c53f2ef63ce05331e2bb72: baseline-arm64-clabbe 2026-03-28 14:14:17,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c53f2ef63ce05331e2bb72 2026-03-28 14:14:17,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:14:17,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c53f2ef63ce05331e2bb72 HTTP/1.1" 200 None 2026-03-28 14:14:17,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:14:17,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 14:14:17,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:14:17,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c53f2ef63ce05331e2bb72 HTTP/1.1" 200 None 2026-03-28 14:14:17,557 - lava_callback - ERROR - Error processing callback for node 69c53f2ef63ce05331e2bb72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37842 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:44:58,731 - lava_callback - INFO - Received callback for node 69c54664f63ce05331e2bbf3 INFO: 172.20.0.1:37846 - "POST /node/69c54664f63ce05331e2bbf3 HTTP/1.0" 202 Accepted 2026-03-28 14:44:58,737 - lava_callback - INFO - Starting async job submit for node 69c54664f63ce05331e2bbf3 2026-03-28 14:44:58,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:44:58,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54664f63ce05331e2bbf3 HTTP/1.1" 200 None 2026-03-28 14:44:58,765 - lava_callback - DEBUG - Processing node 69c54664f63ce05331e2bbf3: baseline-arm-clabbe 2026-03-28 14:44:58,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54664f63ce05331e2bbf3 2026-03-28 14:44:58,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:44:58,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54664f63ce05331e2bbf3 HTTP/1.1" 200 None 2026-03-28 14:44:58,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:44:58,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 14:44:58,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:44:58,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54664f63ce05331e2bbf3 HTTP/1.1" 200 None 2026-03-28 14:44:58,824 - lava_callback - ERROR - Error processing callback for node 69c54664f63ce05331e2bbf3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46976 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:45:38,691 - lava_callback - INFO - Received callback for node 69c5468df63ce05331e2bc07 INFO: 172.20.0.1:46980 - "POST /node/69c5468df63ce05331e2bc07 HTTP/1.0" 202 Accepted 2026-03-28 14:45:38,696 - lava_callback - INFO - Starting async job submit for node 69c5468df63ce05331e2bc07 2026-03-28 14:45:38,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:45:38,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5468df63ce05331e2bc07 HTTP/1.1" 200 None 2026-03-28 14:45:38,712 - lava_callback - DEBUG - Processing node 69c5468df63ce05331e2bc07: baseline-arm-clabbe 2026-03-28 14:45:38,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5468df63ce05331e2bc07 2026-03-28 14:45:38,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:45:38,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5468df63ce05331e2bc07 HTTP/1.1" 200 None 2026-03-28 14:45:38,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:45:38,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 14:45:38,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:45:38,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5468df63ce05331e2bc07 HTTP/1.1" 200 None 2026-03-28 14:45:38,773 - lava_callback - ERROR - Error processing callback for node 69c5468df63ce05331e2bc07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:46:38,769 - lava_callback - INFO - Received callback for node 69c546baf63ce05331e2bc14 INFO: 172.20.0.1:35130 - "POST /node/69c546baf63ce05331e2bc14 HTTP/1.0" 202 Accepted 2026-03-28 14:46:38,774 - lava_callback - INFO - Starting async job submit for node 69c546baf63ce05331e2bc14 2026-03-28 14:46:38,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:46:38,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c546baf63ce05331e2bc14 HTTP/1.1" 200 None 2026-03-28 14:46:38,811 - lava_callback - DEBUG - Processing node 69c546baf63ce05331e2bc14: baseline-arm-clabbe 2026-03-28 14:46:38,812 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c546baf63ce05331e2bc14 2026-03-28 14:46:38,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:46:38,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c546baf63ce05331e2bc14 HTTP/1.1" 200 None 2026-03-28 14:46:38,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:46:38,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 14:46:38,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:46:38,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c546baf63ce05331e2bc14 HTTP/1.1" 200 None 2026-03-28 14:46:38,875 - lava_callback - ERROR - Error processing callback for node 69c546baf63ce05331e2bc14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58204 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:47:18,767 - lava_callback - INFO - Received callback for node 69c546e4f63ce05331e2bc26 INFO: 172.20.0.1:58208 - "POST /node/69c546e4f63ce05331e2bc26 HTTP/1.0" 202 Accepted 2026-03-28 14:47:18,773 - lava_callback - INFO - Starting async job submit for node 69c546e4f63ce05331e2bc26 2026-03-28 14:47:18,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:18,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c546e4f63ce05331e2bc26 HTTP/1.1" 200 None 2026-03-28 14:47:18,790 - lava_callback - DEBUG - Processing node 69c546e4f63ce05331e2bc26: baseline-arm-clabbe 2026-03-28 14:47:18,791 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c546e4f63ce05331e2bc26 2026-03-28 14:47:18,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:18,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c546e4f63ce05331e2bc26 HTTP/1.1" 200 None 2026-03-28 14:47:18,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:18,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 14:47:18,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:18,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c546e4f63ce05331e2bc26 HTTP/1.1" 200 None 2026-03-28 14:47:18,845 - lava_callback - ERROR - Error processing callback for node 69c546e4f63ce05331e2bc26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48466 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:47:58,843 - lava_callback - INFO - Received callback for node 69c5470df63ce05331e2bc2f INFO: 172.20.0.1:48476 - "POST /node/69c5470df63ce05331e2bc2f HTTP/1.0" 202 Accepted 2026-03-28 14:47:58,848 - lava_callback - INFO - Starting async job submit for node 69c5470df63ce05331e2bc2f 2026-03-28 14:47:58,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:58,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5470df63ce05331e2bc2f HTTP/1.1" 200 None 2026-03-28 14:47:58,864 - lava_callback - DEBUG - Processing node 69c5470df63ce05331e2bc2f: baseline-arm-clabbe 2026-03-28 14:47:58,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5470df63ce05331e2bc2f 2026-03-28 14:47:58,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:58,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5470df63ce05331e2bc2f HTTP/1.1" 200 None 2026-03-28 14:47:58,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:58,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-28 14:47:58,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:47:58,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5470df63ce05331e2bc2f HTTP/1.1" 200 None 2026-03-28 14:47:58,916 - lava_callback - ERROR - Error processing callback for node 69c5470df63ce05331e2bc2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:48:32,153 - lava_callback - INFO - Received callback for node 69c54731f63ce05331e2bc41 INFO: 172.20.0.1:50150 - "POST /node/69c54731f63ce05331e2bc41 HTTP/1.0" 202 Accepted 2026-03-28 14:48:32,158 - lava_callback - INFO - Starting async job submit for node 69c54731f63ce05331e2bc41 2026-03-28 14:48:32,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:48:32,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54731f63ce05331e2bc41 HTTP/1.1" 200 None 2026-03-28 14:48:32,893 - lava_callback - DEBUG - Processing node 69c54731f63ce05331e2bc41: baseline-arm-baylibre 2026-03-28 14:48:32,893 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54731f63ce05331e2bc41 2026-03-28 14:48:32,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:48:32,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54731f63ce05331e2bc41 HTTP/1.1" 200 None 2026-03-28 14:48:32,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:48:32,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:48:32,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:48:32,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54731f63ce05331e2bc41 HTTP/1.1" 200 None 2026-03-28 14:48:32,955 - lava_callback - ERROR - Error processing callback for node 69c54731f63ce05331e2bc41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55146 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:49:12,018 - lava_callback - INFO - Received callback for node 69c5475af63ce05331e2bc50 INFO: 172.20.0.1:51462 - "POST /node/69c5475af63ce05331e2bc50 HTTP/1.0" 202 Accepted 2026-03-28 14:49:12,024 - lava_callback - INFO - Starting async job submit for node 69c5475af63ce05331e2bc50 2026-03-28 14:49:12,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:12,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5475af63ce05331e2bc50 HTTP/1.1" 200 None 2026-03-28 14:49:12,040 - lava_callback - DEBUG - Processing node 69c5475af63ce05331e2bc50: baseline-arm-baylibre 2026-03-28 14:49:12,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5475af63ce05331e2bc50 2026-03-28 14:49:12,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:12,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5475af63ce05331e2bc50 HTTP/1.1" 200 None 2026-03-28 14:49:12,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:12,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:49:12,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:12,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5475af63ce05331e2bc50 HTTP/1.1" 200 None 2026-03-28 14:49:12,096 - lava_callback - ERROR - Error processing callback for node 69c5475af63ce05331e2bc50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35196 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:49:52,089 - lava_callback - INFO - Received callback for node 69c54782f63ce05331e2bc5e INFO: 172.20.0.1:50470 - "POST /node/69c54782f63ce05331e2bc5e HTTP/1.0" 202 Accepted 2026-03-28 14:49:52,095 - lava_callback - INFO - Starting async job submit for node 69c54782f63ce05331e2bc5e 2026-03-28 14:49:52,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:52,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54782f63ce05331e2bc5e HTTP/1.1" 200 None 2026-03-28 14:49:52,110 - lava_callback - DEBUG - Processing node 69c54782f63ce05331e2bc5e: baseline-arm-baylibre 2026-03-28 14:49:52,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54782f63ce05331e2bc5e 2026-03-28 14:49:52,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:52,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54782f63ce05331e2bc5e HTTP/1.1" 200 None 2026-03-28 14:49:52,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:52,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:49:52,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:49:52,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54782f63ce05331e2bc5e HTTP/1.1" 200 None 2026-03-28 14:49:52,163 - lava_callback - ERROR - Error processing callback for node 69c54782f63ce05331e2bc5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50266 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:50:32,099 - lava_callback - INFO - Received callback for node 69c547abf63ce05331e2bc6d INFO: 172.20.0.1:34992 - "POST /node/69c547abf63ce05331e2bc6d HTTP/1.0" 202 Accepted 2026-03-28 14:50:32,105 - lava_callback - INFO - Starting async job submit for node 69c547abf63ce05331e2bc6d 2026-03-28 14:50:32,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:50:32,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c547abf63ce05331e2bc6d HTTP/1.1" 200 None 2026-03-28 14:50:32,121 - lava_callback - DEBUG - Processing node 69c547abf63ce05331e2bc6d: baseline-arm-baylibre 2026-03-28 14:50:32,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c547abf63ce05331e2bc6d 2026-03-28 14:50:32,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:50:32,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c547abf63ce05331e2bc6d HTTP/1.1" 200 None 2026-03-28 14:50:32,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:50:32,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:50:32,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:50:32,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c547abf63ce05331e2bc6d HTTP/1.1" 200 None 2026-03-28 14:50:32,177 - lava_callback - ERROR - Error processing callback for node 69c547abf63ce05331e2bc6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46310 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:51:12,214 - lava_callback - INFO - Received callback for node 69c547d5f63ce05331e2bc85 INFO: 172.20.0.1:33704 - "POST /node/69c547d5f63ce05331e2bc85 HTTP/1.0" 202 Accepted 2026-03-28 14:51:12,220 - lava_callback - INFO - Starting async job submit for node 69c547d5f63ce05331e2bc85 2026-03-28 14:51:12,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:12,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c547d5f63ce05331e2bc85 HTTP/1.1" 200 None 2026-03-28 14:51:12,237 - lava_callback - DEBUG - Processing node 69c547d5f63ce05331e2bc85: baseline-arm-baylibre 2026-03-28 14:51:12,237 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c547d5f63ce05331e2bc85 2026-03-28 14:51:12,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:12,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c547d5f63ce05331e2bc85 HTTP/1.1" 200 None 2026-03-28 14:51:12,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:12,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:51:12,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:12,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c547d5f63ce05331e2bc85 HTTP/1.1" 200 None 2026-03-28 14:51:12,297 - lava_callback - ERROR - Error processing callback for node 69c547d5f63ce05331e2bc85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52266 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:51:52,180 - lava_callback - INFO - Received callback for node 69c547fef63ce05331e2bc90 INFO: 172.20.0.1:44086 - "POST /node/69c547fef63ce05331e2bc90 HTTP/1.0" 202 Accepted 2026-03-28 14:51:52,186 - lava_callback - INFO - Starting async job submit for node 69c547fef63ce05331e2bc90 2026-03-28 14:51:52,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:52,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c547fef63ce05331e2bc90 HTTP/1.1" 200 None 2026-03-28 14:51:52,202 - lava_callback - DEBUG - Processing node 69c547fef63ce05331e2bc90: baseline-arm-baylibre 2026-03-28 14:51:52,202 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c547fef63ce05331e2bc90 2026-03-28 14:51:52,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:52,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c547fef63ce05331e2bc90 HTTP/1.1" 200 None 2026-03-28 14:51:52,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:52,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:51:52,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:51:52,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c547fef63ce05331e2bc90 HTTP/1.1" 200 None 2026-03-28 14:51:52,259 - lava_callback - ERROR - Error processing callback for node 69c547fef63ce05331e2bc90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59306 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:53:59,078 - lava_callback - INFO - Received callback for node 69c54878f63ce05331e2bcbd INFO: 172.20.0.1:59322 - "POST /node/69c54878f63ce05331e2bcbd HTTP/1.0" 202 Accepted 2026-03-28 14:53:59,083 - lava_callback - INFO - Starting async job submit for node 69c54878f63ce05331e2bcbd 2026-03-28 14:53:59,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:53:59,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54878f63ce05331e2bcbd HTTP/1.1" 200 None 2026-03-28 14:53:59,100 - lava_callback - DEBUG - Processing node 69c54878f63ce05331e2bcbd: baseline-arm-clabbe 2026-03-28 14:53:59,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54878f63ce05331e2bcbd 2026-03-28 14:53:59,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:53:59,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54878f63ce05331e2bcbd HTTP/1.1" 200 None 2026-03-28 14:53:59,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:53:59,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:53:59,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:53:59,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54878f63ce05331e2bcbd HTTP/1.1" 200 None 2026-03-28 14:53:59,167 - lava_callback - ERROR - Error processing callback for node 69c54878f63ce05331e2bcbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47798 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:55:19,182 - lava_callback - INFO - Received callback for node 69c548c9f63ce05331e2bce1 INFO: 172.20.0.1:47804 - "POST /node/69c548c9f63ce05331e2bce1 HTTP/1.0" 202 Accepted 2026-03-28 14:55:19,188 - lava_callback - INFO - Starting async job submit for node 69c548c9f63ce05331e2bce1 2026-03-28 14:55:19,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:19,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c548c9f63ce05331e2bce1 HTTP/1.1" 200 None 2026-03-28 14:55:19,204 - lava_callback - DEBUG - Processing node 69c548c9f63ce05331e2bce1: baseline-arm-clabbe 2026-03-28 14:55:19,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c548c9f63ce05331e2bce1 2026-03-28 14:55:19,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:19,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c548c9f63ce05331e2bce1 HTTP/1.1" 200 None 2026-03-28 14:55:19,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:19,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:55:19,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:19,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c548c9f63ce05331e2bce1 HTTP/1.1" 200 None 2026-03-28 14:55:19,270 - lava_callback - ERROR - Error processing callback for node 69c548c9f63ce05331e2bce1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60230 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:55:59,320 - lava_callback - INFO - Received callback for node 69c548f2f63ce05331e2bcec INFO: 172.20.0.1:60232 - "POST /node/69c548f2f63ce05331e2bcec HTTP/1.0" 202 Accepted 2026-03-28 14:55:59,326 - lava_callback - INFO - Starting async job submit for node 69c548f2f63ce05331e2bcec 2026-03-28 14:55:59,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:59,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c548f2f63ce05331e2bcec HTTP/1.1" 200 None 2026-03-28 14:55:59,342 - lava_callback - DEBUG - Processing node 69c548f2f63ce05331e2bcec: baseline-arm-clabbe 2026-03-28 14:55:59,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c548f2f63ce05331e2bcec 2026-03-28 14:55:59,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:59,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c548f2f63ce05331e2bcec HTTP/1.1" 200 None 2026-03-28 14:55:59,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:59,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:55:59,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:55:59,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c548f2f63ce05331e2bcec HTTP/1.1" 200 None 2026-03-28 14:55:59,399 - lava_callback - ERROR - Error processing callback for node 69c548f2f63ce05331e2bcec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60362 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:56:32,535 - lava_callback - INFO - Received callback for node 69c54917f63ce05331e2bcfb INFO: 172.20.0.1:54674 - "POST /node/69c54917f63ce05331e2bcfb HTTP/1.0" 202 Accepted 2026-03-28 14:56:32,541 - lava_callback - INFO - Starting async job submit for node 69c54917f63ce05331e2bcfb 2026-03-28 14:56:32,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:56:32,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54917f63ce05331e2bcfb HTTP/1.1" 200 None 2026-03-28 14:56:32,557 - lava_callback - DEBUG - Processing node 69c54917f63ce05331e2bcfb: baseline-arm-baylibre 2026-03-28 14:56:32,557 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54917f63ce05331e2bcfb 2026-03-28 14:56:32,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:56:32,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54917f63ce05331e2bcfb HTTP/1.1" 200 None 2026-03-28 14:56:32,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:56:32,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:56:32,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:56:32,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54917f63ce05331e2bcfb HTTP/1.1" 200 None 2026-03-28 14:56:32,650 - lava_callback - ERROR - Error processing callback for node 69c54917f63ce05331e2bcfb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57528 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:57:59,265 - lava_callback - INFO - Received callback for node 69c5496bf63ce05331e2bd28 INFO: 172.20.0.1:57532 - "POST /node/69c5496bf63ce05331e2bd28 HTTP/1.0" 202 Accepted 2026-03-28 14:57:59,270 - lava_callback - INFO - Starting async job submit for node 69c5496bf63ce05331e2bd28 2026-03-28 14:57:59,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:57:59,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5496bf63ce05331e2bd28 HTTP/1.1" 200 None 2026-03-28 14:57:59,287 - lava_callback - DEBUG - Processing node 69c5496bf63ce05331e2bd28: baseline-arm-clabbe 2026-03-28 14:57:59,287 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5496bf63ce05331e2bd28 2026-03-28 14:57:59,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:57:59,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5496bf63ce05331e2bd28 HTTP/1.1" 200 None 2026-03-28 14:57:59,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:57:59,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:57:59,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:57:59,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5496bf63ce05331e2bd28 HTTP/1.1" 200 None 2026-03-28 14:57:59,340 - lava_callback - ERROR - Error processing callback for node 69c5496bf63ce05331e2bd28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38890 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:58:32,675 - lava_callback - INFO - Received callback for node 69c54991f63ce05331e2bd5e INFO: 172.20.0.1:43090 - "POST /node/69c54991f63ce05331e2bd5e HTTP/1.0" 202 Accepted 2026-03-28 14:58:32,681 - lava_callback - INFO - Starting async job submit for node 69c54991f63ce05331e2bd5e 2026-03-28 14:58:32,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:58:32,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54991f63ce05331e2bd5e HTTP/1.1" 200 None 2026-03-28 14:58:32,697 - lava_callback - DEBUG - Processing node 69c54991f63ce05331e2bd5e: baseline-arm-baylibre 2026-03-28 14:58:32,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54991f63ce05331e2bd5e 2026-03-28 14:58:32,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:58:32,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54991f63ce05331e2bd5e HTTP/1.1" 200 None 2026-03-28 14:58:32,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:58:32,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:58:32,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:58:32,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54991f63ce05331e2bd5e HTTP/1.1" 200 None 2026-03-28 14:58:32,765 - lava_callback - ERROR - Error processing callback for node 69c54991f63ce05331e2bd5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50822 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:59:19,282 - lava_callback - INFO - Received callback for node 69c549bcf63ce05331e2bd6e INFO: 172.20.0.1:50826 - "POST /node/69c549bcf63ce05331e2bd6e HTTP/1.0" 202 Accepted 2026-03-28 14:59:19,288 - lava_callback - INFO - Starting async job submit for node 69c549bcf63ce05331e2bd6e 2026-03-28 14:59:19,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:19,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c549bcf63ce05331e2bd6e HTTP/1.1" 200 None 2026-03-28 14:59:19,303 - lava_callback - DEBUG - Processing node 69c549bcf63ce05331e2bd6e: baseline-arm-clabbe 2026-03-28 14:59:19,303 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c549bcf63ce05331e2bd6e 2026-03-28 14:59:19,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:19,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c549bcf63ce05331e2bd6e HTTP/1.1" 200 None 2026-03-28 14:59:19,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:19,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:59:19,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:19,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c549bcf63ce05331e2bd6e HTTP/1.1" 200 None 2026-03-28 14:59:19,361 - lava_callback - ERROR - Error processing callback for node 69c549bcf63ce05331e2bd6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42514 - "GET / HTTP/1.0" 200 OK 2026-03-28 14:59:59,247 - lava_callback - INFO - Received callback for node 69c549e5f63ce05331e2bd7c INFO: 172.20.0.1:42530 - "POST /node/69c549e5f63ce05331e2bd7c HTTP/1.0" 202 Accepted 2026-03-28 14:59:59,253 - lava_callback - INFO - Starting async job submit for node 69c549e5f63ce05331e2bd7c 2026-03-28 14:59:59,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:59,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c549e5f63ce05331e2bd7c HTTP/1.1" 200 None 2026-03-28 14:59:59,269 - lava_callback - DEBUG - Processing node 69c549e5f63ce05331e2bd7c: baseline-arm-clabbe 2026-03-28 14:59:59,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c549e5f63ce05331e2bd7c 2026-03-28 14:59:59,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:59,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c549e5f63ce05331e2bd7c HTTP/1.1" 200 None 2026-03-28 14:59:59,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:59,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 14:59:59,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 14:59:59,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c549e5f63ce05331e2bd7c HTTP/1.1" 200 None 2026-03-28 14:59:59,322 - lava_callback - ERROR - Error processing callback for node 69c549e5f63ce05331e2bd7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53928 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:00:39,321 - lava_callback - INFO - Received callback for node 69c54a0ef63ce05331e2bd8f INFO: 172.20.0.1:53938 - "POST /node/69c54a0ef63ce05331e2bd8f HTTP/1.0" 202 Accepted 2026-03-28 15:00:39,327 - lava_callback - INFO - Starting async job submit for node 69c54a0ef63ce05331e2bd8f 2026-03-28 15:00:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:00:39,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a0ef63ce05331e2bd8f HTTP/1.1" 200 None 2026-03-28 15:00:39,342 - lava_callback - DEBUG - Processing node 69c54a0ef63ce05331e2bd8f: baseline-arm-clabbe 2026-03-28 15:00:39,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54a0ef63ce05331e2bd8f 2026-03-28 15:00:39,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:00:39,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a0ef63ce05331e2bd8f HTTP/1.1" 200 None 2026-03-28 15:00:39,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:00:39,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 15:00:39,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:00:39,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54a0ef63ce05331e2bd8f HTTP/1.1" 200 None 2026-03-28 15:00:39,398 - lava_callback - ERROR - Error processing callback for node 69c54a0ef63ce05331e2bd8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60160 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:01:19,331 - lava_callback - INFO - Received callback for node 69c54a37f63ce05331e2bd9f INFO: 172.20.0.1:60164 - "POST /node/69c54a37f63ce05331e2bd9f HTTP/1.0" 202 Accepted 2026-03-28 15:01:19,337 - lava_callback - INFO - Starting async job submit for node 69c54a37f63ce05331e2bd9f 2026-03-28 15:01:19,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:19,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a37f63ce05331e2bd9f HTTP/1.1" 200 None 2026-03-28 15:01:19,353 - lava_callback - DEBUG - Processing node 69c54a37f63ce05331e2bd9f: baseline-arm-clabbe 2026-03-28 15:01:19,353 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54a37f63ce05331e2bd9f 2026-03-28 15:01:19,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:19,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a37f63ce05331e2bd9f HTTP/1.1" 200 None 2026-03-28 15:01:19,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:19,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 15:01:19,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:19,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54a37f63ce05331e2bd9f HTTP/1.1" 200 None 2026-03-28 15:01:19,418 - lava_callback - ERROR - Error processing callback for node 69c54a37f63ce05331e2bd9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33512 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:01:59,383 - lava_callback - INFO - Received callback for node 69c54a60f63ce05331e2bdae INFO: 172.20.0.1:33516 - "POST /node/69c54a60f63ce05331e2bdae HTTP/1.0" 202 Accepted 2026-03-28 15:01:59,388 - lava_callback - INFO - Starting async job submit for node 69c54a60f63ce05331e2bdae 2026-03-28 15:01:59,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:59,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a60f63ce05331e2bdae HTTP/1.1" 200 None 2026-03-28 15:01:59,413 - lava_callback - DEBUG - Processing node 69c54a60f63ce05331e2bdae: baseline-arm-clabbe 2026-03-28 15:01:59,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54a60f63ce05331e2bdae 2026-03-28 15:01:59,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:59,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a60f63ce05331e2bdae HTTP/1.1" 200 None 2026-03-28 15:01:59,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:59,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 15:01:59,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:01:59,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54a60f63ce05331e2bdae HTTP/1.1" 200 None 2026-03-28 15:01:59,475 - lava_callback - ERROR - Error processing callback for node 69c54a60f63ce05331e2bdae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39802 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:02:39,389 - lava_callback - INFO - Received callback for node 69c54a89f63ce05331e2bdc7 INFO: 172.20.0.1:39810 - "POST /node/69c54a89f63ce05331e2bdc7 HTTP/1.0" 202 Accepted 2026-03-28 15:02:39,395 - lava_callback - INFO - Starting async job submit for node 69c54a89f63ce05331e2bdc7 2026-03-28 15:02:39,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:02:39,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a89f63ce05331e2bdc7 HTTP/1.1" 200 None 2026-03-28 15:02:39,411 - lava_callback - DEBUG - Processing node 69c54a89f63ce05331e2bdc7: baseline-arm-clabbe 2026-03-28 15:02:39,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54a89f63ce05331e2bdc7 2026-03-28 15:02:39,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:02:39,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54a89f63ce05331e2bdc7 HTTP/1.1" 200 None 2026-03-28 15:02:39,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:02:39,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-28 15:02:39,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:02:39,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54a89f63ce05331e2bdc7 HTTP/1.1" 200 None 2026-03-28 15:02:39,472 - lava_callback - ERROR - Error processing callback for node 69c54a89f63ce05331e2bdc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46760 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:12:33,343 - lava_callback - INFO - Received callback for node 69c54cd5f63ce05331e2be2a INFO: 172.20.0.1:46158 - "POST /node/69c54cd5f63ce05331e2be2a HTTP/1.0" 202 Accepted 2026-03-28 15:12:33,348 - lava_callback - INFO - Starting async job submit for node 69c54cd5f63ce05331e2be2a 2026-03-28 15:12:33,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:12:33,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54cd5f63ce05331e2be2a HTTP/1.1" 200 None 2026-03-28 15:12:33,364 - lava_callback - DEBUG - Processing node 69c54cd5f63ce05331e2be2a: baseline-arm64-baylibre 2026-03-28 15:12:33,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54cd5f63ce05331e2be2a 2026-03-28 15:12:33,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:12:33,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54cd5f63ce05331e2be2a HTTP/1.1" 200 None 2026-03-28 15:12:33,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:12:33,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:12:33,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:12:33,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54cd5f63ce05331e2be2a HTTP/1.1" 200 None 2026-03-28 15:12:33,448 - lava_callback - ERROR - Error processing callback for node 69c54cd5f63ce05331e2be2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:13:53,384 - lava_callback - INFO - Received callback for node 69c54d24f63ce05331e2be3a INFO: 172.20.0.1:60876 - "POST /node/69c54d24f63ce05331e2be3a HTTP/1.0" 202 Accepted 2026-03-28 15:13:53,390 - lava_callback - INFO - Starting async job submit for node 69c54d24f63ce05331e2be3a 2026-03-28 15:13:53,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:13:53,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54d24f63ce05331e2be3a HTTP/1.1" 200 None 2026-03-28 15:13:53,406 - lava_callback - DEBUG - Processing node 69c54d24f63ce05331e2be3a: baseline-arm64-baylibre 2026-03-28 15:13:53,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54d24f63ce05331e2be3a 2026-03-28 15:13:53,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:13:53,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54d24f63ce05331e2be3a HTTP/1.1" 200 None 2026-03-28 15:13:53,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:13:53,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:13:53,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:13:53,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54d24f63ce05331e2be3a HTTP/1.1" 200 None 2026-03-28 15:13:53,467 - lava_callback - ERROR - Error processing callback for node 69c54d24f63ce05331e2be3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42214 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:15:13,512 - lava_callback - INFO - Received callback for node 69c54d72f63ce05331e2be3f INFO: 172.20.0.1:38224 - "POST /node/69c54d72f63ce05331e2be3f HTTP/1.0" 202 Accepted 2026-03-28 15:15:13,517 - lava_callback - INFO - Starting async job submit for node 69c54d72f63ce05331e2be3f 2026-03-28 15:15:13,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:15:13,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54d72f63ce05331e2be3f HTTP/1.1" 200 None 2026-03-28 15:15:13,534 - lava_callback - DEBUG - Processing node 69c54d72f63ce05331e2be3f: baseline-arm64-baylibre 2026-03-28 15:15:13,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54d72f63ce05331e2be3f 2026-03-28 15:15:13,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:15:13,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54d72f63ce05331e2be3f HTTP/1.1" 200 None 2026-03-28 15:15:13,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:15:13,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:15:13,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:15:13,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54d72f63ce05331e2be3f HTTP/1.1" 200 None 2026-03-28 15:15:13,586 - lava_callback - ERROR - Error processing callback for node 69c54d72f63ce05331e2be3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51626 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:16:33,539 - lava_callback - INFO - Received callback for node 69c54dc1f63ce05331e2be4f INFO: 172.20.0.1:59342 - "POST /node/69c54dc1f63ce05331e2be4f HTTP/1.0" 202 Accepted 2026-03-28 15:16:33,545 - lava_callback - INFO - Starting async job submit for node 69c54dc1f63ce05331e2be4f 2026-03-28 15:16:33,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:16:33,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54dc1f63ce05331e2be4f HTTP/1.1" 200 None 2026-03-28 15:16:33,560 - lava_callback - DEBUG - Processing node 69c54dc1f63ce05331e2be4f: baseline-arm64-baylibre 2026-03-28 15:16:33,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54dc1f63ce05331e2be4f 2026-03-28 15:16:33,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:16:33,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54dc1f63ce05331e2be4f HTTP/1.1" 200 None 2026-03-28 15:16:33,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:16:33,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:16:33,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:16:33,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54dc1f63ce05331e2be4f HTTP/1.1" 200 None 2026-03-28 15:16:33,625 - lava_callback - ERROR - Error processing callback for node 69c54dc1f63ce05331e2be4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37260 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:17:53,608 - lava_callback - INFO - Received callback for node 69c54e10f63ce05331e2be5d INFO: 172.20.0.1:40348 - "POST /node/69c54e10f63ce05331e2be5d HTTP/1.0" 202 Accepted 2026-03-28 15:17:53,613 - lava_callback - INFO - Starting async job submit for node 69c54e10f63ce05331e2be5d 2026-03-28 15:17:53,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:17:53,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54e10f63ce05331e2be5d HTTP/1.1" 200 None 2026-03-28 15:17:53,630 - lava_callback - DEBUG - Processing node 69c54e10f63ce05331e2be5d: baseline-arm64-baylibre 2026-03-28 15:17:53,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54e10f63ce05331e2be5d 2026-03-28 15:17:53,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:17:53,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54e10f63ce05331e2be5d HTTP/1.1" 200 None 2026-03-28 15:17:53,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:17:53,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:17:53,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:17:53,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54e10f63ce05331e2be5d HTTP/1.1" 200 None 2026-03-28 15:17:53,689 - lava_callback - ERROR - Error processing callback for node 69c54e10f63ce05331e2be5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53786 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:19:13,652 - lava_callback - INFO - Received callback for node 69c54e5ef63ce05331e2be68 INFO: 172.20.0.1:38048 - "POST /node/69c54e5ef63ce05331e2be68 HTTP/1.0" 202 Accepted 2026-03-28 15:19:13,658 - lava_callback - INFO - Starting async job submit for node 69c54e5ef63ce05331e2be68 2026-03-28 15:19:13,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:19:13,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54e5ef63ce05331e2be68 HTTP/1.1" 200 None 2026-03-28 15:19:13,677 - lava_callback - DEBUG - Processing node 69c54e5ef63ce05331e2be68: baseline-arm64-baylibre 2026-03-28 15:19:13,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54e5ef63ce05331e2be68 2026-03-28 15:19:13,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:19:13,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54e5ef63ce05331e2be68 HTTP/1.1" 200 None 2026-03-28 15:19:13,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:19:13,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:19:13,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:19:13,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54e5ef63ce05331e2be68 HTTP/1.1" 200 None 2026-03-28 15:19:13,747 - lava_callback - ERROR - Error processing callback for node 69c54e5ef63ce05331e2be68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40414 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:20:40,194 - lava_callback - INFO - Received callback for node 69c54eb3f63ce05331e2be7b INFO: 172.20.0.1:48676 - "POST /node/69c54eb3f63ce05331e2be7b HTTP/1.0" 202 Accepted 2026-03-28 15:20:40,200 - lava_callback - INFO - Starting async job submit for node 69c54eb3f63ce05331e2be7b 2026-03-28 15:20:40,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:20:40,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54eb3f63ce05331e2be7b HTTP/1.1" 200 None 2026-03-28 15:20:40,215 - lava_callback - DEBUG - Processing node 69c54eb3f63ce05331e2be7b: baseline-arm64-clabbe 2026-03-28 15:20:40,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54eb3f63ce05331e2be7b 2026-03-28 15:20:40,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:20:40,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54eb3f63ce05331e2be7b HTTP/1.1" 200 None 2026-03-28 15:20:40,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:20:40,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:20:40,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:20:40,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54eb3f63ce05331e2be7b HTTP/1.1" 200 None 2026-03-28 15:20:40,281 - lava_callback - ERROR - Error processing callback for node 69c54eb3f63ce05331e2be7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41654 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:22:00,214 - lava_callback - INFO - Received callback for node 69c54f03f63ce05331e2be89 INFO: 172.20.0.1:42540 - "POST /node/69c54f03f63ce05331e2be89 HTTP/1.0" 202 Accepted 2026-03-28 15:22:00,219 - lava_callback - INFO - Starting async job submit for node 69c54f03f63ce05331e2be89 2026-03-28 15:22:00,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:22:00,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54f03f63ce05331e2be89 HTTP/1.1" 200 None 2026-03-28 15:22:00,258 - lava_callback - DEBUG - Processing node 69c54f03f63ce05331e2be89: baseline-arm64-clabbe 2026-03-28 15:22:00,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54f03f63ce05331e2be89 2026-03-28 15:22:00,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:22:00,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54f03f63ce05331e2be89 HTTP/1.1" 200 None 2026-03-28 15:22:00,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:22:00,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:22:00,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:22:00,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54f03f63ce05331e2be89 HTTP/1.1" 200 None 2026-03-28 15:22:00,336 - lava_callback - ERROR - Error processing callback for node 69c54f03f63ce05331e2be89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42888 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:23:20,313 - lava_callback - INFO - Received callback for node 69c54f53f63ce05331e2be91 INFO: 172.20.0.1:58504 - "POST /node/69c54f53f63ce05331e2be91 HTTP/1.0" 202 Accepted 2026-03-28 15:23:20,319 - lava_callback - INFO - Starting async job submit for node 69c54f53f63ce05331e2be91 2026-03-28 15:23:20,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:23:20,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54f53f63ce05331e2be91 HTTP/1.1" 200 None 2026-03-28 15:23:20,357 - lava_callback - DEBUG - Processing node 69c54f53f63ce05331e2be91: baseline-arm64-clabbe 2026-03-28 15:23:20,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54f53f63ce05331e2be91 2026-03-28 15:23:20,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:23:20,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54f53f63ce05331e2be91 HTTP/1.1" 200 None 2026-03-28 15:23:20,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:23:20,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:23:20,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:23:20,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54f53f63ce05331e2be91 HTTP/1.1" 200 None 2026-03-28 15:23:20,409 - lava_callback - ERROR - Error processing callback for node 69c54f53f63ce05331e2be91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46678 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:24:40,317 - lava_callback - INFO - Received callback for node 69c54fa4f63ce05331e2be9e INFO: 172.20.0.1:58810 - "POST /node/69c54fa4f63ce05331e2be9e HTTP/1.0" 202 Accepted 2026-03-28 15:24:40,323 - lava_callback - INFO - Starting async job submit for node 69c54fa4f63ce05331e2be9e 2026-03-28 15:24:40,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:24:40,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54fa4f63ce05331e2be9e HTTP/1.1" 200 None 2026-03-28 15:24:40,339 - lava_callback - DEBUG - Processing node 69c54fa4f63ce05331e2be9e: baseline-arm64-clabbe 2026-03-28 15:24:40,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54fa4f63ce05331e2be9e 2026-03-28 15:24:40,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:24:40,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54fa4f63ce05331e2be9e HTTP/1.1" 200 None 2026-03-28 15:24:40,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:24:40,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:24:40,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:24:40,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54fa4f63ce05331e2be9e HTTP/1.1" 200 None 2026-03-28 15:24:40,396 - lava_callback - ERROR - Error processing callback for node 69c54fa4f63ce05331e2be9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35540 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:26:00,370 - lava_callback - INFO - Received callback for node 69c54ff3f63ce05331e2beac INFO: 172.20.0.1:60754 - "POST /node/69c54ff3f63ce05331e2beac HTTP/1.0" 202 Accepted 2026-03-28 15:26:00,376 - lava_callback - INFO - Starting async job submit for node 69c54ff3f63ce05331e2beac 2026-03-28 15:26:00,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:26:00,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54ff3f63ce05331e2beac HTTP/1.1" 200 None 2026-03-28 15:26:00,392 - lava_callback - DEBUG - Processing node 69c54ff3f63ce05331e2beac: baseline-arm64-clabbe 2026-03-28 15:26:00,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c54ff3f63ce05331e2beac 2026-03-28 15:26:00,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:26:00,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c54ff3f63ce05331e2beac HTTP/1.1" 200 None 2026-03-28 15:26:00,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:26:00,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:26:00,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:26:00,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c54ff3f63ce05331e2beac HTTP/1.1" 200 None 2026-03-28 15:26:00,449 - lava_callback - ERROR - Error processing callback for node 69c54ff3f63ce05331e2beac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38114 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:27:00,440 - lava_callback - INFO - Received callback for node 69c55041f63ce05331e2beb4 INFO: 172.20.0.1:53054 - "POST /node/69c55041f63ce05331e2beb4 HTTP/1.0" 202 Accepted 2026-03-28 15:27:00,446 - lava_callback - INFO - Starting async job submit for node 69c55041f63ce05331e2beb4 2026-03-28 15:27:00,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:27:00,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55041f63ce05331e2beb4 HTTP/1.1" 200 None 2026-03-28 15:27:00,461 - lava_callback - DEBUG - Processing node 69c55041f63ce05331e2beb4: baseline-arm64-clabbe 2026-03-28 15:27:00,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55041f63ce05331e2beb4 2026-03-28 15:27:00,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:27:00,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55041f63ce05331e2beb4 HTTP/1.1" 200 None 2026-03-28 15:27:00,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:27:00,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-28 15:27:00,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:27:00,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55041f63ce05331e2beb4 HTTP/1.1" 200 None 2026-03-28 15:27:00,527 - lava_callback - ERROR - Error processing callback for node 69c55041f63ce05331e2beb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40808 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:28:14,257 - lava_callback - INFO - Received callback for node 69c55087f63ce05331e2bec2 INFO: 172.20.0.1:41564 - "POST /node/69c55087f63ce05331e2bec2 HTTP/1.0" 202 Accepted 2026-03-28 15:28:14,262 - lava_callback - INFO - Starting async job submit for node 69c55087f63ce05331e2bec2 2026-03-28 15:28:14,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:28:14,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55087f63ce05331e2bec2 HTTP/1.1" 200 None 2026-03-28 15:28:14,278 - lava_callback - DEBUG - Processing node 69c55087f63ce05331e2bec2: baseline-arm64-baylibre 2026-03-28 15:28:14,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55087f63ce05331e2bec2 2026-03-28 15:28:14,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:28:14,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55087f63ce05331e2bec2 HTTP/1.1" 200 None 2026-03-28 15:28:14,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:28:14,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:28:14,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:28:14,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55087f63ce05331e2bec2 HTTP/1.1" 200 None 2026-03-28 15:28:14,339 - lava_callback - ERROR - Error processing callback for node 69c55087f63ce05331e2bec2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37818 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:32:14,395 - lava_callback - INFO - Received callback for node 69c5516bf63ce05331e2bedb INFO: 172.20.0.1:53316 - "POST /node/69c5516bf63ce05331e2bedb HTTP/1.0" 202 Accepted 2026-03-28 15:32:14,400 - lava_callback - INFO - Starting async job submit for node 69c5516bf63ce05331e2bedb 2026-03-28 15:32:14,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:32:14,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5516bf63ce05331e2bedb HTTP/1.1" 200 None 2026-03-28 15:32:14,416 - lava_callback - DEBUG - Processing node 69c5516bf63ce05331e2bedb: baseline-arm64-baylibre 2026-03-28 15:32:14,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5516bf63ce05331e2bedb 2026-03-28 15:32:14,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:32:14,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5516bf63ce05331e2bedb HTTP/1.1" 200 None 2026-03-28 15:32:14,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:32:14,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:32:14,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:32:14,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5516bf63ce05331e2bedb HTTP/1.1" 200 None 2026-03-28 15:32:14,484 - lava_callback - ERROR - Error processing callback for node 69c5516bf63ce05331e2bedb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40064 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:33:34,497 - lava_callback - INFO - Received callback for node 69c551bbf63ce05331e2beed INFO: 172.20.0.1:56040 - "POST /node/69c551bbf63ce05331e2beed HTTP/1.0" 202 Accepted 2026-03-28 15:33:34,502 - lava_callback - INFO - Starting async job submit for node 69c551bbf63ce05331e2beed 2026-03-28 15:33:34,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:33:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c551bbf63ce05331e2beed HTTP/1.1" 200 None 2026-03-28 15:33:34,518 - lava_callback - DEBUG - Processing node 69c551bbf63ce05331e2beed: baseline-arm64-baylibre 2026-03-28 15:33:34,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c551bbf63ce05331e2beed 2026-03-28 15:33:34,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:33:34,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c551bbf63ce05331e2beed HTTP/1.1" 200 None 2026-03-28 15:33:34,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:33:34,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:33:34,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:33:34,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c551bbf63ce05331e2beed HTTP/1.1" 200 None 2026-03-28 15:33:34,592 - lava_callback - ERROR - Error processing callback for node 69c551bbf63ce05331e2beed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53806 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:34:34,495 - lava_callback - INFO - Received callback for node 69c55208f63ce05331e2bf05 INFO: 172.20.0.1:44148 - "POST /node/69c55208f63ce05331e2bf05 HTTP/1.0" 202 Accepted 2026-03-28 15:34:34,501 - lava_callback - INFO - Starting async job submit for node 69c55208f63ce05331e2bf05 2026-03-28 15:34:34,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:34:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55208f63ce05331e2bf05 HTTP/1.1" 200 None 2026-03-28 15:34:34,518 - lava_callback - DEBUG - Processing node 69c55208f63ce05331e2bf05: baseline-arm64-baylibre 2026-03-28 15:34:34,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55208f63ce05331e2bf05 2026-03-28 15:34:34,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:34:34,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55208f63ce05331e2bf05 HTTP/1.1" 200 None 2026-03-28 15:34:34,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:34:34,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:34:34,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:34:34,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55208f63ce05331e2bf05 HTTP/1.1" 200 None 2026-03-28 15:34:34,575 - lava_callback - ERROR - Error processing callback for node 69c55208f63ce05331e2bf05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52808 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:38:34,761 - lava_callback - INFO - Received callback for node 69c552ebf63ce05331e2bf2d INFO: 172.20.0.1:45150 - "POST /node/69c552ebf63ce05331e2bf2d HTTP/1.0" 202 Accepted 2026-03-28 15:38:34,767 - lava_callback - INFO - Starting async job submit for node 69c552ebf63ce05331e2bf2d 2026-03-28 15:38:34,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:38:34,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c552ebf63ce05331e2bf2d HTTP/1.1" 200 None 2026-03-28 15:38:34,784 - lava_callback - DEBUG - Processing node 69c552ebf63ce05331e2bf2d: baseline-arm64-baylibre 2026-03-28 15:38:34,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c552ebf63ce05331e2bf2d 2026-03-28 15:38:34,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:38:34,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c552ebf63ce05331e2bf2d HTTP/1.1" 200 None 2026-03-28 15:38:34,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:38:34,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:38:34,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:38:34,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c552ebf63ce05331e2bf2d HTTP/1.1" 200 None 2026-03-28 15:38:34,847 - lava_callback - ERROR - Error processing callback for node 69c552ebf63ce05331e2bf2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55416 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:40:01,002 - lava_callback - INFO - Received callback for node 69c5533ef63ce05331e2bf3e INFO: 172.20.0.1:45826 - "POST /node/69c5533ef63ce05331e2bf3e HTTP/1.0" 202 Accepted 2026-03-28 15:40:01,008 - lava_callback - INFO - Starting async job submit for node 69c5533ef63ce05331e2bf3e 2026-03-28 15:40:01,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:40:01,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5533ef63ce05331e2bf3e HTTP/1.1" 200 None 2026-03-28 15:40:01,035 - lava_callback - DEBUG - Processing node 69c5533ef63ce05331e2bf3e: baseline-arm64-clabbe 2026-03-28 15:40:01,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5533ef63ce05331e2bf3e 2026-03-28 15:40:01,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:40:01,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5533ef63ce05331e2bf3e HTTP/1.1" 200 None 2026-03-28 15:40:01,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:40:01,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:40:01,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:40:01,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5533ef63ce05331e2bf3e HTTP/1.1" 200 None 2026-03-28 15:40:01,100 - lava_callback - ERROR - Error processing callback for node 69c5533ef63ce05331e2bf3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45830 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:41:21,030 - lava_callback - INFO - Received callback for node 69c5538df63ce05331e2bf49 INFO: 172.20.0.1:43378 - "POST /node/69c5538df63ce05331e2bf49 HTTP/1.0" 202 Accepted 2026-03-28 15:41:21,035 - lava_callback - INFO - Starting async job submit for node 69c5538df63ce05331e2bf49 2026-03-28 15:41:21,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:41:21,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5538df63ce05331e2bf49 HTTP/1.1" 200 None 2026-03-28 15:41:21,051 - lava_callback - DEBUG - Processing node 69c5538df63ce05331e2bf49: baseline-arm64-clabbe 2026-03-28 15:41:21,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5538df63ce05331e2bf49 2026-03-28 15:41:21,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:41:21,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5538df63ce05331e2bf49 HTTP/1.1" 200 None 2026-03-28 15:41:21,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:41:21,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:41:21,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:41:21,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5538df63ce05331e2bf49 HTTP/1.1" 200 None 2026-03-28 15:41:21,123 - lava_callback - ERROR - Error processing callback for node 69c5538df63ce05331e2bf49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2026-03-28 15:42:15,117 - lava_callback - INFO - Received callback for node 69c553d5f63ce05331e2bf59 INFO: 172.20.0.1:34766 - "POST /node/69c553d5f63ce05331e2bf59 HTTP/1.0" 202 Accepted 2026-03-28 15:42:15,123 - lava_callback - INFO - Starting async job submit for node 69c553d5f63ce05331e2bf59 2026-03-28 15:42:15,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:42:15,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c553d5f63ce05331e2bf59 HTTP/1.1" 200 None 2026-03-28 15:42:15,138 - lava_callback - DEBUG - Processing node 69c553d5f63ce05331e2bf59: baseline-arm64-baylibre 2026-03-28 15:42:15,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c553d5f63ce05331e2bf59 2026-03-28 15:42:15,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:42:15,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c553d5f63ce05331e2bf59 HTTP/1.1" 200 None 2026-03-28 15:42:15,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:34768 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 15:42:15,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:42:15,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:42:15,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c553d5f63ce05331e2bf59 HTTP/1.1" 200 None 2026-03-28 15:42:15,190 - lava_callback - ERROR - Error processing callback for node 69c553d5f63ce05331e2bf59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55354 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:43:41,124 - lava_callback - INFO - Received callback for node 69c55426f63ce05331e2bf70 INFO: 172.20.0.1:53112 - "POST /node/69c55426f63ce05331e2bf70 HTTP/1.0" 202 Accepted 2026-03-28 15:43:41,130 - lava_callback - INFO - Starting async job submit for node 69c55426f63ce05331e2bf70 2026-03-28 15:43:41,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:43:41,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55426f63ce05331e2bf70 HTTP/1.1" 200 None 2026-03-28 15:43:41,147 - lava_callback - DEBUG - Processing node 69c55426f63ce05331e2bf70: baseline-arm64-clabbe 2026-03-28 15:43:41,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55426f63ce05331e2bf70 2026-03-28 15:43:41,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:43:41,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55426f63ce05331e2bf70 HTTP/1.1" 200 None 2026-03-28 15:43:41,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:43:41,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:43:41,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:43:41,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55426f63ce05331e2bf70 HTTP/1.1" 200 None 2026-03-28 15:43:41,227 - lava_callback - ERROR - Error processing callback for node 69c55426f63ce05331e2bf70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45650 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:45:01,160 - lava_callback - INFO - Received callback for node 69c55475f63ce05331e2bf7a INFO: 172.20.0.1:54246 - "POST /node/69c55475f63ce05331e2bf7a HTTP/1.0" 202 Accepted 2026-03-28 15:45:01,166 - lava_callback - INFO - Starting async job submit for node 69c55475f63ce05331e2bf7a 2026-03-28 15:45:01,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:45:01,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55475f63ce05331e2bf7a HTTP/1.1" 200 None 2026-03-28 15:45:01,181 - lava_callback - DEBUG - Processing node 69c55475f63ce05331e2bf7a: baseline-arm64-clabbe 2026-03-28 15:45:01,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55475f63ce05331e2bf7a 2026-03-28 15:45:01,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:45:01,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55475f63ce05331e2bf7a HTTP/1.1" 200 None 2026-03-28 15:45:01,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:45:01,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:45:01,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:45:01,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55475f63ce05331e2bf7a HTTP/1.1" 200 None 2026-03-28 15:45:01,254 - lava_callback - ERROR - Error processing callback for node 69c55475f63ce05331e2bf7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34246 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:46:21,217 - lava_callback - INFO - Received callback for node 69c554c2f63ce05331e2bf8a INFO: 172.20.0.1:47464 - "POST /node/69c554c2f63ce05331e2bf8a HTTP/1.0" 202 Accepted 2026-03-28 15:46:21,223 - lava_callback - INFO - Starting async job submit for node 69c554c2f63ce05331e2bf8a 2026-03-28 15:46:21,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:46:21,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c554c2f63ce05331e2bf8a HTTP/1.1" 200 None 2026-03-28 15:46:21,271 - lava_callback - DEBUG - Processing node 69c554c2f63ce05331e2bf8a: baseline-arm64-clabbe 2026-03-28 15:46:21,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c554c2f63ce05331e2bf8a 2026-03-28 15:46:21,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:46:21,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c554c2f63ce05331e2bf8a HTTP/1.1" 200 None 2026-03-28 15:46:21,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:46:21,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:46:21,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:46:21,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c554c2f63ce05331e2bf8a HTTP/1.1" 200 None 2026-03-28 15:46:21,324 - lava_callback - ERROR - Error processing callback for node 69c554c2f63ce05331e2bf8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43222 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:47:41,250 - lava_callback - INFO - Received callback for node 69c55510f63ce05331e2bf98 INFO: 172.20.0.1:37328 - "POST /node/69c55510f63ce05331e2bf98 HTTP/1.0" 202 Accepted 2026-03-28 15:47:41,256 - lava_callback - INFO - Starting async job submit for node 69c55510f63ce05331e2bf98 2026-03-28 15:47:41,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:47:41,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55510f63ce05331e2bf98 HTTP/1.1" 200 None 2026-03-28 15:47:41,272 - lava_callback - DEBUG - Processing node 69c55510f63ce05331e2bf98: baseline-arm64-clabbe 2026-03-28 15:47:41,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55510f63ce05331e2bf98 2026-03-28 15:47:41,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:47:41,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55510f63ce05331e2bf98 HTTP/1.1" 200 None 2026-03-28 15:47:41,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:47:41,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:47:41,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:47:41,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55510f63ce05331e2bf98 HTTP/1.1" 200 None 2026-03-28 15:47:41,325 - lava_callback - ERROR - Error processing callback for node 69c55510f63ce05331e2bf98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57508 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:49:01,341 - lava_callback - INFO - Received callback for node 69c5555ff63ce05331e2bfaa INFO: 172.20.0.1:45792 - "POST /node/69c5555ff63ce05331e2bfaa HTTP/1.0" 202 Accepted 2026-03-28 15:49:01,347 - lava_callback - INFO - Starting async job submit for node 69c5555ff63ce05331e2bfaa 2026-03-28 15:49:01,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:49:01,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5555ff63ce05331e2bfaa HTTP/1.1" 200 None 2026-03-28 15:49:01,363 - lava_callback - DEBUG - Processing node 69c5555ff63ce05331e2bfaa: baseline-arm64-clabbe 2026-03-28 15:49:01,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5555ff63ce05331e2bfaa 2026-03-28 15:49:01,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:49:01,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5555ff63ce05331e2bfaa HTTP/1.1" 200 None 2026-03-28 15:49:01,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:49:01,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:49:01,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:49:01,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5555ff63ce05331e2bfaa HTTP/1.1" 200 None 2026-03-28 15:49:01,423 - lava_callback - ERROR - Error processing callback for node 69c5555ff63ce05331e2bfaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:50:21,398 - lava_callback - INFO - Received callback for node 69c555acf63ce05331e2bfbe INFO: 172.20.0.1:54370 - "POST /node/69c555acf63ce05331e2bfbe HTTP/1.0" 202 Accepted 2026-03-28 15:50:21,404 - lava_callback - INFO - Starting async job submit for node 69c555acf63ce05331e2bfbe 2026-03-28 15:50:21,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:50:21,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c555acf63ce05331e2bfbe HTTP/1.1" 200 None 2026-03-28 15:50:21,419 - lava_callback - DEBUG - Processing node 69c555acf63ce05331e2bfbe: baseline-arm64-clabbe 2026-03-28 15:50:21,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c555acf63ce05331e2bfbe 2026-03-28 15:50:21,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:50:21,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c555acf63ce05331e2bfbe HTTP/1.1" 200 None 2026-03-28 15:50:21,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:50:21,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:50:21,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:50:21,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c555acf63ce05331e2bfbe HTTP/1.1" 200 None 2026-03-28 15:50:21,474 - lava_callback - ERROR - Error processing callback for node 69c555acf63ce05331e2bfbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:51:41,556 - lava_callback - INFO - Received callback for node 69c555faf63ce05331e2bfc6 INFO: 172.20.0.1:46846 - "POST /node/69c555faf63ce05331e2bfc6 HTTP/1.0" 202 Accepted 2026-03-28 15:51:41,561 - lava_callback - INFO - Starting async job submit for node 69c555faf63ce05331e2bfc6 2026-03-28 15:51:41,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:51:41,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c555faf63ce05331e2bfc6 HTTP/1.1" 200 None 2026-03-28 15:51:41,578 - lava_callback - DEBUG - Processing node 69c555faf63ce05331e2bfc6: baseline-arm64-clabbe 2026-03-28 15:51:41,578 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c555faf63ce05331e2bfc6 2026-03-28 15:51:41,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:51:41,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c555faf63ce05331e2bfc6 HTTP/1.1" 200 None 2026-03-28 15:51:41,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:51:41,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:51:41,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:51:41,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c555faf63ce05331e2bfc6 HTTP/1.1" 200 None 2026-03-28 15:51:41,638 - lava_callback - ERROR - Error processing callback for node 69c555faf63ce05331e2bfc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:53:01,478 - lava_callback - INFO - Received callback for node 69c55648f63ce05331e2bfda INFO: 172.20.0.1:60566 - "POST /node/69c55648f63ce05331e2bfda HTTP/1.0" 202 Accepted 2026-03-28 15:53:01,483 - lava_callback - INFO - Starting async job submit for node 69c55648f63ce05331e2bfda 2026-03-28 15:53:01,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:53:01,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55648f63ce05331e2bfda HTTP/1.1" 200 None 2026-03-28 15:53:01,499 - lava_callback - DEBUG - Processing node 69c55648f63ce05331e2bfda: baseline-arm64-clabbe 2026-03-28 15:53:01,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55648f63ce05331e2bfda 2026-03-28 15:53:01,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:53:01,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55648f63ce05331e2bfda HTTP/1.1" 200 None 2026-03-28 15:53:01,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:53:01,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:53:01,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:53:01,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55648f63ce05331e2bfda HTTP/1.1" 200 None 2026-03-28 15:53:01,568 - lava_callback - ERROR - Error processing callback for node 69c55648f63ce05331e2bfda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37890 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:54:01,521 - lava_callback - INFO - Received callback for node 69c55695f63ce05331e2bfe4 INFO: 172.20.0.1:40292 - "POST /node/69c55695f63ce05331e2bfe4 HTTP/1.0" 202 Accepted 2026-03-28 15:54:01,527 - lava_callback - INFO - Starting async job submit for node 69c55695f63ce05331e2bfe4 2026-03-28 15:54:01,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:54:01,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55695f63ce05331e2bfe4 HTTP/1.1" 200 None 2026-03-28 15:54:01,543 - lava_callback - DEBUG - Processing node 69c55695f63ce05331e2bfe4: baseline-arm64-clabbe 2026-03-28 15:54:01,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55695f63ce05331e2bfe4 2026-03-28 15:54:01,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:54:01,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55695f63ce05331e2bfe4 HTTP/1.1" 200 None 2026-03-28 15:54:01,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:54:01,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:54:01,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:54:01,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55695f63ce05331e2bfe4 HTTP/1.1" 200 None 2026-03-28 15:54:01,612 - lava_callback - ERROR - Error processing callback for node 69c55695f63ce05331e2bfe4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41944 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:55:21,599 - lava_callback - INFO - Received callback for node 69c556e4f63ce05331e2bff7 INFO: 172.20.0.1:44288 - "POST /node/69c556e4f63ce05331e2bff7 HTTP/1.0" 202 Accepted 2026-03-28 15:55:21,604 - lava_callback - INFO - Starting async job submit for node 69c556e4f63ce05331e2bff7 2026-03-28 15:55:21,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:55:21,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c556e4f63ce05331e2bff7 HTTP/1.1" 200 None 2026-03-28 15:55:21,632 - lava_callback - DEBUG - Processing node 69c556e4f63ce05331e2bff7: baseline-arm64-clabbe 2026-03-28 15:55:21,632 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c556e4f63ce05331e2bff7 2026-03-28 15:55:21,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:55:21,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c556e4f63ce05331e2bff7 HTTP/1.1" 200 None 2026-03-28 15:55:21,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:55:21,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-28 15:55:21,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:55:21,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c556e4f63ce05331e2bff7 HTTP/1.1" 200 None 2026-03-28 15:55:21,723 - lava_callback - ERROR - Error processing callback for node 69c556e4f63ce05331e2bff7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41852 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:56:35,717 - lava_callback - INFO - Received callback for node 69c5572df63ce05331e2c004 INFO: 172.20.0.1:40100 - "POST /node/69c5572df63ce05331e2c004 HTTP/1.0" 202 Accepted 2026-03-28 15:56:35,723 - lava_callback - INFO - Starting async job submit for node 69c5572df63ce05331e2c004 2026-03-28 15:56:35,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:56:35,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5572df63ce05331e2c004 HTTP/1.1" 200 None 2026-03-28 15:56:35,743 - lava_callback - DEBUG - Processing node 69c5572df63ce05331e2c004: baseline-arm64-baylibre 2026-03-28 15:56:35,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5572df63ce05331e2c004 2026-03-28 15:56:35,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:56:35,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5572df63ce05331e2c004 HTTP/1.1" 200 None 2026-03-28 15:56:35,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:56:35,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 15:56:35,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:56:35,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5572df63ce05331e2c004 HTTP/1.1" 200 None 2026-03-28 15:56:35,818 - lava_callback - ERROR - Error processing callback for node 69c5572df63ce05331e2c004: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36730 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:57:55,795 - lava_callback - INFO - Received callback for node 69c5577df63ce05331e2c016 INFO: 172.20.0.1:37572 - "POST /node/69c5577df63ce05331e2c016 HTTP/1.0" 202 Accepted 2026-03-28 15:57:55,801 - lava_callback - INFO - Starting async job submit for node 69c5577df63ce05331e2c016 2026-03-28 15:57:55,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:57:55,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5577df63ce05331e2c016 HTTP/1.1" 200 None 2026-03-28 15:57:55,819 - lava_callback - DEBUG - Processing node 69c5577df63ce05331e2c016: baseline-arm64-baylibre 2026-03-28 15:57:55,819 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5577df63ce05331e2c016 2026-03-28 15:57:55,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:57:55,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5577df63ce05331e2c016 HTTP/1.1" 200 None 2026-03-28 15:57:55,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:57:55,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 15:57:55,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:57:55,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5577df63ce05331e2c016 HTTP/1.1" 200 None 2026-03-28 15:57:55,882 - lava_callback - ERROR - Error processing callback for node 69c5577df63ce05331e2c016: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47610 - "GET / HTTP/1.0" 200 OK 2026-03-28 15:59:21,745 - lava_callback - INFO - Received callback for node 69c557cff63ce05331e2c02d INFO: 172.20.0.1:56246 - "POST /node/69c557cff63ce05331e2c02d HTTP/1.0" 202 Accepted 2026-03-28 15:59:21,750 - lava_callback - INFO - Starting async job submit for node 69c557cff63ce05331e2c02d 2026-03-28 15:59:21,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:59:21,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c557cff63ce05331e2c02d HTTP/1.1" 200 None 2026-03-28 15:59:21,766 - lava_callback - DEBUG - Processing node 69c557cff63ce05331e2c02d: baseline-arm64-clabbe 2026-03-28 15:59:21,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c557cff63ce05331e2c02d 2026-03-28 15:59:21,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:59:21,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c557cff63ce05331e2c02d HTTP/1.1" 200 None 2026-03-28 15:59:21,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:59:21,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 15:59:21,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 15:59:21,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c557cff63ce05331e2c02d HTTP/1.1" 200 None 2026-03-28 15:59:21,831 - lava_callback - ERROR - Error processing callback for node 69c557cff63ce05331e2c02d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55834 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:00:41,834 - lava_callback - INFO - Received callback for node 69c5581ff63ce05331e2c03b INFO: 172.20.0.1:49424 - "POST /node/69c5581ff63ce05331e2c03b HTTP/1.0" 202 Accepted 2026-03-28 16:00:41,844 - lava_callback - INFO - Starting async job submit for node 69c5581ff63ce05331e2c03b 2026-03-28 16:00:41,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:00:41,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5581ff63ce05331e2c03b HTTP/1.1" 200 None 2026-03-28 16:00:41,877 - lava_callback - DEBUG - Processing node 69c5581ff63ce05331e2c03b: baseline-arm64-clabbe 2026-03-28 16:00:41,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5581ff63ce05331e2c03b 2026-03-28 16:00:41,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:00:41,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5581ff63ce05331e2c03b HTTP/1.1" 200 None 2026-03-28 16:00:41,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:00:41,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:00:41,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:00:41,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5581ff63ce05331e2c03b HTTP/1.1" 200 None 2026-03-28 16:00:41,941 - lava_callback - ERROR - Error processing callback for node 69c5581ff63ce05331e2c03b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46334 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:02:01,901 - lava_callback - INFO - Received callback for node 69c5586ef63ce05331e2c04d INFO: 172.20.0.1:32974 - "POST /node/69c5586ef63ce05331e2c04d HTTP/1.0" 202 Accepted 2026-03-28 16:02:01,906 - lava_callback - INFO - Starting async job submit for node 69c5586ef63ce05331e2c04d 2026-03-28 16:02:01,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:02:01,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5586ef63ce05331e2c04d HTTP/1.1" 200 None 2026-03-28 16:02:01,927 - lava_callback - DEBUG - Processing node 69c5586ef63ce05331e2c04d: baseline-arm64-clabbe 2026-03-28 16:02:01,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5586ef63ce05331e2c04d 2026-03-28 16:02:01,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:02:01,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5586ef63ce05331e2c04d HTTP/1.1" 200 None 2026-03-28 16:02:01,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:02:01,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:02:01,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:02:01,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5586ef63ce05331e2c04d HTTP/1.1" 200 None 2026-03-28 16:02:01,986 - lava_callback - ERROR - Error processing callback for node 69c5586ef63ce05331e2c04d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51354 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:03:21,946 - lava_callback - INFO - Received callback for node 69c558baf63ce05331e2c064 INFO: 172.20.0.1:41340 - "POST /node/69c558baf63ce05331e2c064 HTTP/1.0" 202 Accepted 2026-03-28 16:03:21,951 - lava_callback - INFO - Starting async job submit for node 69c558baf63ce05331e2c064 2026-03-28 16:03:21,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:03:21,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c558baf63ce05331e2c064 HTTP/1.1" 200 None 2026-03-28 16:03:21,967 - lava_callback - DEBUG - Processing node 69c558baf63ce05331e2c064: baseline-arm64-clabbe 2026-03-28 16:03:21,967 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c558baf63ce05331e2c064 2026-03-28 16:03:21,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:03:22,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c558baf63ce05331e2c064 HTTP/1.1" 200 None 2026-03-28 16:03:22,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:03:22,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:03:22,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:03:22,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c558baf63ce05331e2c064 HTTP/1.1" 200 None 2026-03-28 16:03:22,049 - lava_callback - ERROR - Error processing callback for node 69c558baf63ce05331e2c064: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46130 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:04:41,966 - lava_callback - INFO - Received callback for node 69c55908f63ce05331e2c06f INFO: 172.20.0.1:60540 - "POST /node/69c55908f63ce05331e2c06f HTTP/1.0" 202 Accepted 2026-03-28 16:04:41,971 - lava_callback - INFO - Starting async job submit for node 69c55908f63ce05331e2c06f 2026-03-28 16:04:41,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:04:41,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55908f63ce05331e2c06f HTTP/1.1" 200 None 2026-03-28 16:04:41,987 - lava_callback - DEBUG - Processing node 69c55908f63ce05331e2c06f: baseline-arm64-clabbe 2026-03-28 16:04:41,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55908f63ce05331e2c06f 2026-03-28 16:04:41,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:04:42,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55908f63ce05331e2c06f HTTP/1.1" 200 None 2026-03-28 16:04:42,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:04:42,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:04:42,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:04:42,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55908f63ce05331e2c06f HTTP/1.1" 200 None 2026-03-28 16:04:42,044 - lava_callback - ERROR - Error processing callback for node 69c55908f63ce05331e2c06f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46434 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:06:02,053 - lava_callback - INFO - Received callback for node 69c55956f63ce05331e2c085 INFO: 172.20.0.1:50846 - "POST /node/69c55956f63ce05331e2c085 HTTP/1.0" 202 Accepted 2026-03-28 16:06:02,058 - lava_callback - INFO - Starting async job submit for node 69c55956f63ce05331e2c085 2026-03-28 16:06:02,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:06:02,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55956f63ce05331e2c085 HTTP/1.1" 200 None 2026-03-28 16:06:02,074 - lava_callback - DEBUG - Processing node 69c55956f63ce05331e2c085: baseline-arm64-clabbe 2026-03-28 16:06:02,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55956f63ce05331e2c085 2026-03-28 16:06:02,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:06:02,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55956f63ce05331e2c085 HTTP/1.1" 200 None 2026-03-28 16:06:02,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:06:02,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:06:02,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:06:02,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55956f63ce05331e2c085 HTTP/1.1" 200 None 2026-03-28 16:06:02,131 - lava_callback - ERROR - Error processing callback for node 69c55956f63ce05331e2c085: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49874 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 16:07:16,325 - lava_callback - INFO - Received callback for node 69c5599ef63ce05331e2c090 INFO: 172.20.0.1:49884 - "POST /node/69c5599ef63ce05331e2c090 HTTP/1.0" 202 Accepted 2026-03-28 16:07:16,331 - lava_callback - INFO - Starting async job submit for node 69c5599ef63ce05331e2c090 2026-03-28 16:07:16,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:07:16,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5599ef63ce05331e2c090 HTTP/1.1" 200 None 2026-03-28 16:07:16,346 - lava_callback - DEBUG - Processing node 69c5599ef63ce05331e2c090: baseline-arm64-baylibre 2026-03-28 16:07:16,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5599ef63ce05331e2c090 2026-03-28 16:07:16,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:07:16,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5599ef63ce05331e2c090 HTTP/1.1" 200 None 2026-03-28 16:07:16,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:07:16,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:07:16,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:07:16,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5599ef63ce05331e2c090 HTTP/1.1" 200 None 2026-03-28 16:07:16,429 - lava_callback - ERROR - Error processing callback for node 69c5599ef63ce05331e2c090: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46450 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:08:22,142 - lava_callback - INFO - Received callback for node 69c559f1f63ce05331e2c0a2 INFO: 172.20.0.1:41614 - "POST /node/69c559f1f63ce05331e2c0a2 HTTP/1.0" 202 Accepted 2026-03-28 16:08:22,147 - lava_callback - INFO - Starting async job submit for node 69c559f1f63ce05331e2c0a2 2026-03-28 16:08:22,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:08:22,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c559f1f63ce05331e2c0a2 HTTP/1.1" 200 None 2026-03-28 16:08:22,166 - lava_callback - DEBUG - Processing node 69c559f1f63ce05331e2c0a2: baseline-arm64-clabbe 2026-03-28 16:08:22,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c559f1f63ce05331e2c0a2 2026-03-28 16:08:22,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:08:22,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c559f1f63ce05331e2c0a2 HTTP/1.1" 200 None 2026-03-28 16:08:22,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:08:22,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:08:22,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:08:22,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c559f1f63ce05331e2c0a2 HTTP/1.1" 200 None 2026-03-28 16:08:22,219 - lava_callback - ERROR - Error processing callback for node 69c559f1f63ce05331e2c0a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33296 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:09:42,197 - lava_callback - INFO - Received callback for node 69c55a3ff63ce05331e2c0b8 INFO: 172.20.0.1:37900 - "POST /node/69c55a3ff63ce05331e2c0b8 HTTP/1.0" 202 Accepted 2026-03-28 16:09:42,203 - lava_callback - INFO - Starting async job submit for node 69c55a3ff63ce05331e2c0b8 2026-03-28 16:09:42,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:09:42,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55a3ff63ce05331e2c0b8 HTTP/1.1" 200 None 2026-03-28 16:09:42,221 - lava_callback - DEBUG - Processing node 69c55a3ff63ce05331e2c0b8: baseline-arm64-clabbe 2026-03-28 16:09:42,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55a3ff63ce05331e2c0b8 2026-03-28 16:09:42,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:09:42,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55a3ff63ce05331e2c0b8 HTTP/1.1" 200 None 2026-03-28 16:09:42,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:09:42,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:09:42,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:09:42,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55a3ff63ce05331e2c0b8 HTTP/1.1" 200 None 2026-03-28 16:09:42,277 - lava_callback - ERROR - Error processing callback for node 69c55a3ff63ce05331e2c0b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38340 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:11:02,223 - lava_callback - INFO - Received callback for node 69c55a8df63ce05331e2c0bf INFO: 172.20.0.1:35294 - "POST /node/69c55a8df63ce05331e2c0bf HTTP/1.0" 202 Accepted 2026-03-28 16:11:02,229 - lava_callback - INFO - Starting async job submit for node 69c55a8df63ce05331e2c0bf 2026-03-28 16:11:02,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:11:02,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55a8df63ce05331e2c0bf HTTP/1.1" 200 None 2026-03-28 16:11:02,244 - lava_callback - DEBUG - Processing node 69c55a8df63ce05331e2c0bf: baseline-arm64-clabbe 2026-03-28 16:11:02,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55a8df63ce05331e2c0bf 2026-03-28 16:11:02,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:11:02,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55a8df63ce05331e2c0bf HTTP/1.1" 200 None 2026-03-28 16:11:02,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:11:02,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:11:02,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:11:02,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55a8df63ce05331e2c0bf HTTP/1.1" 200 None 2026-03-28 16:11:02,319 - lava_callback - ERROR - Error processing callback for node 69c55a8df63ce05331e2c0bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34146 - "GET / HTTP/1.0" 200 OK 2026-03-28 16:12:22,320 - lava_callback - INFO - Received callback for node 69c55ad9f63ce05331e2c0d0 INFO: 172.20.0.1:38632 - "POST /node/69c55ad9f63ce05331e2c0d0 HTTP/1.0" 202 Accepted 2026-03-28 16:12:22,326 - lava_callback - INFO - Starting async job submit for node 69c55ad9f63ce05331e2c0d0 2026-03-28 16:12:22,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:12:22,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55ad9f63ce05331e2c0d0 HTTP/1.1" 200 None 2026-03-28 16:12:22,342 - lava_callback - DEBUG - Processing node 69c55ad9f63ce05331e2c0d0: baseline-arm64-clabbe 2026-03-28 16:12:22,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c55ad9f63ce05331e2c0d0 2026-03-28 16:12:22,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:12:22,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55ad9f63ce05331e2c0d0 HTTP/1.1" 200 None 2026-03-28 16:12:22,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:12:22,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-28 16:12:22,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 16:12:22,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c55ad9f63ce05331e2c0d0 HTTP/1.1" 200 None 2026-03-28 16:12:22,395 - lava_callback - ERROR - Error processing callback for node 69c55ad9f63ce05331e2c0d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]